New Technologies for Future VLBI Correlators B. Carlson

1. New Technologies for Future VLBI CorrelatorsB. Carlson High Res. Astronomy, June 8-12, 2003 Socorro NM

2. Outline • Key architecture and design issues. • Review of correlator groups/developments. • EVLA Correlator for VLBI. • Design challenges. • Design methodologies. • Design tools. • “Vision for the Future” High Res. Astronomy - New Technologies for Future VLBI Correlators

3. Key Issues • Architecture: determined by requirements • BBC configurations: are parallel processing methods required? • Bandwidth: directly affects digital clock rate, choice of technology, cost. • Number of antennas: small…large N, may affect flexibility, signal processing choices/tradeoffs. • Number of spectral channels. Flexibility requirements determine how spectral channels are allocated to bandwidth. High Res. Astronomy - New Technologies for Future VLBI Correlators

4. Key Issues • Architecture cont’d: • Dynamic range: affects number of bits used at various stages. Can have a major impact on the choice of architecture. • Baseline, real-time/fiber delays, recording based: affects delay tracking hardware complexity/cost. • Flexibility: specific antennas/BBC configurations OR wide range of different antenna handling abilities? • Number of bits in correlator: can affect where digital filtering (if any) is performed (i.e. at antenna or at correlator). High Res. Astronomy - New Technologies for Future VLBI Correlators

5. Key Issues • Architecture cont’d: • Delay and phase tracking at the antenna: simplifies correlator design; more complex H/W and S/W at antenna; “got one chance to get it right”; some say YES, some say NO. High Res. Astronomy - New Technologies for Future VLBI Correlators

6. Key Issues • H/W design issues: determined by architecture and speed requirements • Low, medium, high-speed. Transmission lines? Propagation delays on circuit boards important? Signal integrity? Power dissipation? • Station-to-baseline signal distribution: critical at high speeds, easier at medium to low speeds. May or may not be issue. • H/W or Software correlator: determined by bandwidth, number of antennas, number of channels, duty cycle. • Delay and phase tracking: dynamic range, precision requirements, drive how these are handled. High Res. Astronomy - New Technologies for Future VLBI Correlators

7. Key Issues • S/W design issues: determined by architecture and flexibility requirements. • “Observing modes”-based design? • Goal/science-oriented design? • Data handling performance requirements: how fast does data have to be dumped? How many channels/baseline? High Res. Astronomy - New Technologies for Future VLBI Correlators

8. Correlator Groups + Recent Developments • XF Correlators • MkIV/JIVE/… (Haystack/European cons.) – 1k…4k channels/baseline; 2 bits; 64 MHz clock rate. • Specifically built for VLBI. High Res. Astronomy - New Technologies for Future VLBI Correlators

9. Correlator Groups + Recent Developments • FX Correlators • ATA (SETI Institute/UC Berkeley) – ~1k chans/baseline; 4 bits; 350 antennas(!) (60k baselines); 100(?) MHz clock. • ATNF research into ALMA (J. Bunton) • Japanese ALMA 2GC proposal – ~>10k(?) chans/baseline; 4 bits; 64 antennas; 125(?) MHz clock High Res. Astronomy - New Technologies for Future VLBI Correlators

10. Correlator Groups + Recent Developments • FX Correlators cont’d • (All) use poly-phase FIR/FFT filterbank for superior performance over previous generation (VLBA, Mitaka) FX correlators. • FX design may limit flexibility in handling BBs, and assignment of channels to multiple spectral regions of interest. • Rely on re-quantization to 4 bits after FFT to deal with station-to-baseline bandwidth problem…could limit dynamic range depending on how this is done. • Not particularly built for VLBI, but could handle VLBI with front-end delay/phase compensation. • Possibly the optimum solution for large-N correlation. High Res. Astronomy - New Technologies for Future VLBI Correlators

11. Correlator Groups + Recent Developments • Hybrid Correlators (digital filter…XF correlation) • EVLA “WIDAR” – NRC Canada – 16k…4M chans/baseline; 4/8-bit; 40+ antennas; high dynamic range (50-60 dB); 256 MHz clock • U.S. ALMA – NRAO – recently proposed to be upgraded to hybrid capability – up to 32k chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock. • European ALMA 2GC proposal – ASTRON et. al. – 16k…32k(?) chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock • (All) perform digital filtering followed by XF correlation. • Various dynamic range capabilities. • Various flexibilities, “ways of doing things”. • EVLA correlator is only one with VLBI capability “out of the box”…U.S. ALMA could with front-end phase tracking…European ALMA has fundamental ability if so chosen… High Res. Astronomy - New Technologies for Future VLBI Correlators

12. Correlator Groups + Recent Developments • Software Correlators • Australia (S. Tingay et. al.) – Swinburne University of Technology parallel supercomputer: 180, 2 GHz PCs. (XF?); a few antennas; modest bandwidth; low duty cycle. • NRAO (T. Morgan) – feasibility study using the Cray X1 supercomputer. May have advantages over PC cluster S/W correlators because of “matrix math unit” that allows bit-level operations. FX. Too expensive for EVLA, but for smaller-scale VLBI correlator…? • LOFAR – ASTRON/NRL/Haystack – FFT (spatial) beamforming or FX correlator?. 32 MHz. Mixture of H/W and S/W elements? High Res. Astronomy - New Technologies for Future VLBI Correlators

13. EVLA Correlator for VLBI—Overview • Bandwidth, flexibility, and signal processing method makes it suitable/useable for VLBI. • Uses VLBI-standard sample rates. • Can handle antennas with a mixture of bandwidths w/ or w/o delay/phase tracking at antennas. • Recording-based or real-time VLBI (fiber/VSI interfaces). High Res. Astronomy - New Technologies for Future VLBI Correlators

14. EVLA Correlator for VLBI—Overview • Sub-band multi-beaming for multi-source observing/in-beam phase referencing. • Tradeoff bandwidth for number of antennas/beams w/o rewiring the correlator. • 16k chans/baseline…4M chans/baseline w/ recirculation. • 1, 2, 3, 4, 8-bit capability. High-speed, 2x1000 phase bins • Digital phased-EVLA output. High Res. Astronomy - New Technologies for Future VLBI Correlators

15. EVLA Correlator—Station Brd Processing • Flexible BBC handling: 2 BBCs @ 2 GHz each…32 BBCs <=128 MHz each. • 0.25 second delay buffer using inexpensive SDRAM…complex design. • All-digital +/- 1/16th sample delay tracking (wideband…narrowband) • 32 independent digital filters…bandwidth, beam, location in BB. Probably implement in gate array. • Phase-cal extraction, quantizer statistics. • 2 Pulsar timers for gating/binning. • Data path switching. • VSI input for connection to new-generation VLBI recorders. • VSI output for recording sub-band data (EVLA Phase-II reqs)…could use board as filterbank board at antenna if desired. High Res. Astronomy - New Technologies for Future VLBI Correlators

16. EVLA Correlator—Baseline Brd Processing • 64, 2048 complex-lag correlator chips: 16 cells @ 128 lags/cell; fringe-stopping; high performance readout for recirculation + phase binning. • Corr. chip design complete: 4 Mgates; est 2.7 W PD; est $400k NRE, $90/chip in 10k qty, $50/chip in 25k qty. • Recirculation by up to a factor of 256 for up to 256k chans/correlation (4M chans/baseline). • High performance Long-Term Accumulator (LTA). • Output UDP/IP frames containing data…for switching to backend. High Res. Astronomy - New Technologies for Future VLBI Correlators

17. EVLA Correlator Configurations • NRC will deliver a 32-station correlator (16 GHz/station) • For EVLA Phase-I this leaves 5 “spares”: 5 stations @ 16 GHz; 10 stations at 4 GHz (4k chans/baseline); 20 stations @ 1 GHz (1k chans/baseline). • This extra capacity could be used for the VLBA (+some control S/W). • EVLA Phase-II • NRAO will upgrade to 40 stations. • 27 EVLA + 10 NMA…leaves 3 spares (3 stations…6 stations…12 stations) High Res. Astronomy - New Technologies for Future VLBI Correlators

18. High Res. Astronomy - New Technologies for Future VLBI Correlators

19. Design Challenges • Speed • Most if not all signals require transmission lines. • Power dissipation is everything. • can affect choice of technology: FPGA or ASIC + feature size. • can limit functionality: “what can be put on a chip”. • Complexity • Astronomers want “everything”. • Requires increasingly complex H/W functions for performance. • Moore’s Law yeah, but with increasing speed and complexity, everything gets more difficult for engineers. High Res. Astronomy - New Technologies for Future VLBI Correlators

20. High Res. Astronomy - New Technologies for Future VLBI Correlators

21. High Res. Astronomy - New Technologies for Future VLBI Correlators

22. Design Methodologies • To cope with increasing speed and complexity: • HDL/RTL design for chip logic (“code that describes hardware”) • Incremental code and test…hierarchical design. • Much faster than schematic-based design. • Code can be written in technology-independent fashion to target the “latest and greatest” chips. Code can be re-used… • “Testbenches” for testing can take as long to write as the logic function itself! • Signal integrity and timing analysis through the development cycle. • High performance ($$), constraint-based PAR (Place And Route) tools both for chips and circuit boards. • Post-PAR simulation, to verify signal integrity and timing margins…pretty darn sure its going to work before PCB fabricated. High Res. Astronomy - New Technologies for Future VLBI Correlators

23. Design Tools • We are using “Mentor Graphics” • FPGA Advantage: complete suite of tools for chip HDL+graphical hierarchical design, simulation, and “synthesis”. • “Expedition” PCB tools: • “Signal Vision” for transmission line “what-if” analysis. • Schematic capture…constraints entry. • PCB constraint-based PAR: unbelievable capability. • Post-PAR signal integrity analysis, timing margin analysis. • Thermal analysis. • Cost: about $250k for the EVLA correlator team; ~$40k/yr maintenance…already finding it is well worth the cost. High Res. Astronomy - New Technologies for Future VLBI Correlators

24. “Vision for the Future” • Maximum bandwidth possible. • Moderate size N. • ~10k or so channels/baseline. • Disk-based and eventually fiber-connected VLBI. • Flexibility, able to “absorb” outputs from variety of antennas (GBT? DSN? EVN? LAR?) • Delay and phase tracking at the correlator (my choice, my guess). • EVLA and VLBA merge into one instrument, into one correlator. • EVLA correlator suited to meet all of these requirements for the foreseeable future… High Res. Astronomy - New Technologies for Future VLBI Correlators