SMT702 and SMT712 Clocks and Reset

1. SMT702 and SMT712Clocks and Reset Synchronous Data Acquisition and Output

2. SMT702 Operation • Clock Circuitry • PLL+VCO ensures stable fixed sampling frequency, maxed to 1.5GHz (Fs=3GHz) • Possible to output Clock/2 = 750MHz (Fs=1.5GHz) • No External clock needed this way, but can be chosen through software • Reference clock can be external or 10MHz/100MHz from backplane (software selected) • Clock period from ADC is Fs/4 (1.5GHz/4 = 375MHz)

3. ADC Sampling Phase Relationship It is sometimes important to sample the two channels in phase - • Counters begin in an unknown state at reset, so may not be synchronous • Resets are applied at same time, but relative phase must be verified • ADC sampling clock is split into FPGA with one input being phase shifted • Phase shift occurs in Fs T/255 increments, providing a good image of clock edge • Latched clock shape is stored in the SMT702/SMT712 configuration and status registers to be read by Host • Host issues reset if clocks are out of tolerance in phase to each other

5. Host Controlled Steps to Synchronous Reset • ADC reset is issued from Host • FSM issues a reset to ADC’s, DCM’s, and all logic depending on these clocks • After reset from Host to ADC is released, DCM’s and logic are held in reset until clock is available from ADC’s • When DCM’s lock onto ADC clock, all logic resets are released

7. DCM A is phase shifted 255 times from Host • The clock provided by each ADC through a BUFG is latched with this clock, providing a good image of the clock edge of one full period of each clock • The Host reads these 255 values (‘1’ or ‘0’) and counts (XOR’s) the values between each sampled clock to find out how far off the phase relationship is between each Fs • If it is within tolerance the loop quits, otherwise another reset is issued and the clock is scanned again (phase shifted)

8. Final Phase-shift is for Data Capture • Once the ADC’s are aligned to each other, the clock needs to be phase shifted again • The clock needs to be aligned correctly to the capture window of the data • Because they are both in sync, the same clock can be used for both ADC’s • External clock requirements • The default value in the software works with the onboard VCO+PLL circuitry • A different frequency needs a different phase shift • Trial and error recapture

9. SMT712 Operation • Clock Circuitry • PLL+VCO ensures a stable fixed sampling frequency, maxed to 2.3GHz • An additional clock out of clock circuitry is provided for system synchronization (e.g. SMT702 synchronization) • No external clock needed this way, but can be chosen through software • Reference clock can be external or 10MHz/100MHz from backplane (software selected) • Clock period from DAC is Fs/8 (2.3GHz/8 = 287.5MHz)

10. DAC Clock Phase Relationship • Special attention is required for sending data in phase • There is no reset to the DAC’s • An internal clock counter/divider provides the 1/8 clock to the FPGA • No reset means counters start in an unknown state • Counters are matched by removing clock cycles from the internal counter • A high speed ‘AND’ gate provides this feature

12. Synchronization Pulse • An FSM manages pulses to the DAC clock counters • The FSM is clocked by a DCM clock adjusted to 8/7 of the DAC clock period (328.6MHz) • When a request is sent for synchronization, a pulse equal to one clock period is sent to the ‘AND’ gate input of the DAC. This equates to 7 clock pulses to the internal counter, essentially stopping the counter for 7 increments. • Clock edge is sampled again and loop quits if clock correction is complete

13. Clocked DAC A and B Phase • Clock edge is sampled the same way as the SMT702 • Sampling clock for each DAC is split into a BUFG and a DCM • The DCM is phase shifted over a whole period 255 times • Latched edge is stored to SMT702/SMT712 configuration and staus registers to be read by Host • A software ‘XOR’ of these samples determines relative phase • Because there is no reset, the Host instead issues a force synchronization pulse

16. Final Phase-shift for Data Alignment • Once the DAC’s clocks are aligned to each other, the clocks are then phase shifted again • The clocks need to be corrected to properly latch the data out of to the DAC • Because they are both in sync the same clock can be used for both DAC’s • External Clock Requirements • The default value for data alignment in the software works with the onboard VCO+PLL circuitry • A different sampling frequency would need a different phase shift

17. Questions?

18. SMT702 and SMT712DDR2 Memory Interface smtxxx_ddr2_interface.ngc FIFO Wrapper

19. smtXXX_ddr2_interface.ngc • The provided netlist is a wrapper modelling a FIFO, making memory transactions transparent • It provides an interface to the Xilinx CoreGenerator MIG 2.0 • The best place for information on the MIG is the Xilinx provided datasheets • The provided mechanism for writing and reading from memory behaves as a FIFO would, with DDR data • The usual expected FIFO flags and lines are available to the top of the component • Write clock, Read clock • Reset • Data in, Data out • Empty, Full • Write enable, Read enable • Etc. • The difference in behaviour from a FIFO is reading and writing at the same time • The wrapper has small FIFO’s inside the netlist that allow for some leeway in this respect, but to avoid complications, the interface should be viewed as a ‘duplex’ system. (i.e. read or write at one time, but not both)

21. Questions?

22. SMT702 and SMT712Host Application Environment SMT7026

23. Board Discovery • SmtGetBoardInfo(UINT nIndex, SMTBI& info); • SmtGetBoardIndex(UINT nBaseAddress); struct SMTBI { SMTBoardType Type; // Define the board type char cszType[32]; // String type of board UINT nBase; // Base address of the board UINT nRange; // IO range used by the board SMTHWStatusHwStatus; SMTLockLockStatus; SMTOpenOpenRes; };

24. Board Discovery These functions and structures recover the base address (BAR 1) of the first SMT board discovered on the PXIe bus. The application scans the PXIe bus and steps the pointer until the SMT702 or SMT712 is discovered, then the device is opened. For the SMT702, If there are multiple boards of the same type discovered on the PXIe bus, the pointer will store each of the recovered base addresses and open the subsequent boards each in turn. The BAR1 address of the PXIe device is the base address of user configurable space on the board, where our design lies. The TOC for the XLink interface blocks lie at offset 0x1000 from BAR1. Most of the addresses/functions used in the systems are read or written to the lower addresses of the BAR1 base address, starting at offset 0x400. (e.g.cb_smt702_reg.vhd or cb_smt712_reg.vhd) The offset requirements are handled by the specific SMT7x2 libraries, but if it is desired to add additional functionality it may be important to know these address offsets.

25. The definitions for these registers (cb_smt702_reg.vhd and cb_smt712_reg.vhd) can be found in the User Guide for each system and contain status and configuration settings for the various components and clock settings. Smt7xx.lib Functions The configuration and operation of both the SMT702 and SMT712 depends on the library Smt7xx.lib for all its basic functions. (If the graph is desired, the additional SmtGraph.lib is needed) On top of these functions are wrappers defined in two more libraries (sources are built in the provided workspace), one for the SMT702, and one for the SMT712 The Smt7xx.lib library is actually a wrapper itself for our more general SMT functions defined in the default SMT library.

26. XLink Functions Available in Library HXDEV XOpen (DEV_CLASS DevClass, UINT DevNo, void *buf, UINT bytes)= 0; Void XRead (HXDEV h, void *buf, UINT nBytes) = 0; Void XWrite (HXDEV h, void *buf, UINT nBytes) = 0; Void XRead (HXDEV h, void *buf, UINT bytes, double &MBps) = 0; Void XWrite (HXDEV h, void *buf, UINT bytes, double &MBps) = 0; Void XClose (HXDEV h) = 0; Void XCancel (DEV_CLASS DevClass, UINT DevNo) = 0; Memory Functions void * MemLock (UINT nBytes, void *pBuf, MDLEntry * pEntries, UINT &nEntCnt) = 0; void MemUnlock (void *pMem) = 0;

27. Read and Write Functions to Control and Status Registers void Read (unsigned intnReg, unsigned int &nRd) = 0; void ReadBuf (unsigned intnReg, DWORD *pBuf, unsigned intnSize) = 0; void Write (unsigned intnReg, unsigned intnCmd) = 0;

28. Smt702Config The board specific register information and settings for the SMT702 can be found in the Smt702_reg.h and their descriptions are defined in the User Guide. Nearly all of these software defined registers relate directly to the cb_smt702_reg.vhd registers in the firmware. • Steps to Data Acquisition with GUI • The steps to making a valid data capture can be summed as: • Set Power Supplies • Apply Resets • Serial Interface Programming • DCM Adjustments • DDR2 Memory Storage • Playback

29. Setting the Power Supplies • ADC A • ADC B • Clock Circuitry • OnButtonPwrApply() • On pressing ‘Apply’, the GUI first issues a reset to all the components using the ‘Reset()’ function • The system monitor which gathers information on the die temperature, voltage, etc. is then initialize using ‘SetSysMonitor()’ • ‘SetPower()’ then sets the enable bit in the command register to turn on the power supply for each of the components (clock circuitry and DAC’s are in power off state at start up)

30. Issuing Resets ADCs DCM’s DDR2 Banks A and B SHB 1 SHB 2 • OnButtonRstApply() • On pressing ‘Apply’, the GUI issues a reset to all the components above which are checked using the ‘Reset()’ function. These are not cleared automatically.

31. Programming the Serial Interfaces • Reference Source • Clock Source • ADC A Configuration Register Programming • ADC B Configuration Register Programming • ADC’s Calibration • OnButtonIntApply() • First the reference source bit is set using SetRefClk() • The default clock settings are then sent to the clock registers in the FPGA. Once these values have been verified in a read back, the values are then ‘updated’, meaning they are sent to the clock chip over a serial link. This is all accomplished with SetClkSrc() • Poll() is used to monitor the completion bit of this serial programming • The default ADC A settings are then sent to the ADC A registers in the FPGA. Once these values have been verified in a read back, the values are then updated and serial programmed also. This is all accomplished with SetADC() • Poll() is used to monitor the completion bit of this serial programming

32. Programming the Serial Interfaces • The default ADC B settings are then sent to the ADC B registers in the FPGA. Once these values have been verified in a read back, the values are then updated and serial programmed also. This is all accomplished with SetADC() • Poll() is used to monitor the completion bit of this serial programming • The next step is a calibration setting for the both of the ADC’s issued with SetCalibration() • Poll() is used to monitor the completion bit of this calibration • The final step is a reset issued to ADC A and B with Reset()

33. DCM Shift Adjustments • ADC A Shift Adjustment(value) • ADC B shift Adjustment(value) • OnButtonShiftedadjApply() • The first SetShiftAdj() is issued to initialize the DCM’s to a phase offset of 0. The completion of this is monitored using Poll() • A loop is then begun until the phase relationship between ADC A sampling clock and ADC B sampling clock are close enough in phase to be considered synchronized • First a reset is sent to the ADC’s using Reset() • SetShiftAdj() is applied 128 times to cover a half period of the clock, ensuring the clock edge recovery. Read() recovers the clock edge of each sampled phaseshift and XOR’s the result • If the result is high, a counter is incremented. If the counter is too high (phase is too misaligned) the loop repeats • Once the clock phases are close enough, a final SetShiftAdj() is issued to align the clock edge to data window correctly for data capture. The default value for a 3Gsps sampling clock is set already.

34. Filling DDR2 Memory • Fill up DDR banks • OnButtonDDR2MemApply() • First the DDR2 is issued a reset using Write(), then cleared using Write() again. • Poll() monitors all the initialization steps in the DDR of both banks to monitor when the memory is available for use • The DDR2 capture enable bit is set using Write() • A loop until is begun • The FIFO A and FIFO B full flags are Read() until the memory is full, then the loop exits • The capture enable bit for the DDR2 is then cleared with Write()

35. Acquisitions • Save Capture Data to XX files per channel • OnButtonAcqStart() • A thread is begun to read the memory contents (m_hStart) • A 2K buffer of memory is locked on the host using MemLock() • The number of bits stored in FIFO A is Read() • The number of bits stored in FIFO B is Read() • The buffer is then filled with 2K samples from FIFO A using CpRead(). These samples are masked and bit shifted to represent the separate samples (they arrive concatenated). • The temporary buffer is then stored to file (if selected) by using GetDatatoFile() first to store ADC A’s samples, then again to append the 2K samples to the last read 2K samples. • Samples from ADC A are then sent to the graphing functions

36. Acquisitions • The buffer is then filled with 2K samples from FIFO B using CpRead(). These samples are masked and bit shifted to represent the separate samples (they arrive concatenated). • The temporary buffer is then stored to file (if selected) by using GetDatatoFile() first to store ADC B’s samples, then again to append the 2K samples to the last read 2K samples. • Samples from ADC B are then sent to the graphing functions • The thread continues until ‘Stop’ is pressed (m_hStop) or the memory is emptied

37. Smt712Config The board specific register information and settings for the SMT712 can be found in the Smt712_reg.h and their descriptions are defined in the User Guide. Nearly all of these software defined registers relate directly to the cb_smt712_reg.vhd registers in the firmware. • Steps to Data Transmission with GUI • The steps to making a valid data transmission can be summed as: • Set Power Supplies • Apply Appropriate Resets • Serial Interfaces • DCM Adjustments • Data Source Selection

38. Setting the Power Supplies • ADC A • ADC B • Clock Circuitry • OnButtonPwrApply() • On pressing ‘Apply’, the GUI first issues a reset to all the components using the ‘Reset()’ function • The synchronization enable, reset, and synch start bit are then cleared with Write() • Power to the clock circuitry and DAC’s are then cycled off then on using PwrOff() and PwrOn(), if the options are checked.

39. Issuing Resets DACs Clock Circuitry DDR2 Banks A and B System Monitor SHB 1 SHB 2 • OnButtonRstApply() • On pressing ‘Apply’, the GUI enables the synchronization (does not send a reset) with SetDacSync(). • A reset is then applied to all the check options with Reset() • A high is sent to the synchronization bit using Synch() • The data source is selected for transmitting using the SetDataSrc() function

40. Programming the Serial Interfaces • Reference Source • Clock Source • DAC A Configuration Register Programming • DAC B Configuration Register Programming • Synchronize DAC’s • OnButtonIntApply() • First the reference source bit is set using SetRefClk() • The reset bit for the DAC’s is then cleared using Write() • The default DAC A settings are then sent to the DAC A registers in the FPGA. Once these values have been verified in a read back, the values are then updated and serial programmed. This is all accomplished with SetDAC() • The default DAC B settings are then sent to the DAC B registers in the FPGA. Once these values have been verified in a read back, the values are then updated and serial programmed also. This is all accomplished with SetDAC() • The reset for the clock circuitry is cleared using Write() • Synch() sends a high to the synchronization bit

41. Programming the Serial Interfaces • The default clock settings are then sent to the clock registers in the FPGA and verified in a read back. This is all accomplished with SetClkSrc() • The clock register settings are then serial programmed to the clock chip with an update to the FPGA registers using UpdateClkReg() • After the clock chip is initialized, a calibration sequence setting for the VCO is sent to the FPGA with CalibrateVCO(). UpdateClkReg() is again used to update and send these serial configuration words to the clock chip. • Read() is used to determine when this calibration cycle is complete. • Synch() sends a low, then a high to the synchronization bit. • A reset is then sent to DCM A with Rst_DCMDAC(), then to DCM B

42. Programming the Serial Interfaces • If a synchronization of the DAC’s is checked – • The synch reset bit will be cleared with Write() • The first SetShiftAdj() is issued to initialize the DCM’s to a phase offset of 0. • A loop then begins – • Write() sets the synchronization enable bit, then clears it • SetShiftAdj() is applied 255 times to cover a half period of the clock, ensuring the clock edge recovery. The function returns the XOR’ed result of the clock edge of each sampled phaseshift • If the result is high, a counter is incremented. If the counter is too high (phase is too misaligned) the loop repeats

43. DCM Shift Adjustments • DAC A Shift Adjustment(value) • DAC B shift Adjustment(value) • OnButtonShiftedadjApply() • SetShiftAdj() is issued to align the clock to the correct window time for accurate latching of the data to the DAC. The default value is set to correctly work with the onboard clock circuitry

44. Data Source Selection • File (DDR2 Pattern Generator, both DAC A and B • DDR2 Pattern Generator (specify size of pattern, number of periods) • FPGA embedded DDS (specify phase increment) • SHB First – Write() clears the start/stop bit for DDR2 enable • If User File to DDR2 is Source~ • The specified file for A is scanned for the number of samples it holds with GetFileSize(), then the same is performed for channel B. • Write() sets the DDR2 reset, then clears the reset • The DDR2 is chosen as the source for DAC output with SetDataSrc() • The size of the pattern for DAC A is written with Write(), then the size of the pattern for B is written with Write() • The actual pattern in the file for channel A is scanned and written to the DDR2 A offset with SendPattern(). The same is again done for channel B.

45. Data Source Selection • If DDR2 Pattern Generator is Source~ • First Write() sets the DDR2 reset, then clears the reset • The DDR2 is chosen as the source for DAC output with SetDataSrc() • The size of the pattern for DAC A is written with Write(), then the size of the pattern for B is written with Write() • A sine wave is then generated in the Host with the size/number of periods specified. The pattern is sent to channel A DDR2 over XLink using CpWrite(), then channel B DDR2.

46. Data Source Selection • If embedded DDS is Source~ • The DDS is chosen as the source for DAC output with SetDataSrc() • The phase increment variable is sent to the DDS for DAC A with Write(), then the phase increment is sent to the DDS for DAC B • If SHB is Source~ • No special configuration required - SHB input is simply chosen as the source for DAC output with SetDataSrc()

47. Questions?