1 / 14

Digital Packaging Processor - Overview Gordon Hurford Nov 7, 2011

Digital Packaging Processor - Overview Gordon Hurford Nov 7, 2011. EOVSA Technical Design Meeting - NJIT. Digital Packaging Processor. Role of DPP Overall assumptions and priorities Interface Overview Tasks and algorithms Hardware/software Implementation

wilda
Download Presentation

Digital Packaging Processor - Overview Gordon Hurford Nov 7, 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Digital Packaging Processor - OverviewGordon HurfordNov 7, 2011 EOVSA Technical Design Meeting - NJIT

  2. Digital Packaging Processor • Role of DPP • Overall assumptions and priorities • Interface Overview • Tasks and algorithms • Hardware/software Implementation ----------------------------------------------------------- • DPP Interface Details

  3. Data System Approach Fundamental Drivers • Very limited software resources • Non-trivial data rate and volume • Automated analysis pipeline for efficient observing • Must have science-useable system in place by September 2013 • Data products to be readily useable by broader solar community • Data products with preset parameters • Data products with user-selected parameters • Tools and support for experienced users

  4. Data System Approach Implications • Monomode observing • Calibrated data archived in application-specific databases • Reliance on existing software packages • Miriad package for calibration & mapping • RHESSI SolarSoft package for user interface and data product display • RHESSI database model • Err on side of over-rejection of data • Limited initial support for ‘nice-to-have’ options • Limited initial support for calibration refinements • Limited support for non-solar applications

  5. Data System Assumptions • All information required for data analysis is written by the DPP to the Interim Data Base • Engineering data acquisition, archiving and display is the responsibility of the ACC, and is “largely” decoupled from science data.

  6. Nomenclature • Data frame = Interval representing data from one correlator cycle (20 ms, ~4000 channels with 500 MHz range) • Spectral frame = Data corresponding to a complete frequency-agile cycle (nominal 1 second, 10s to 100’s of ‘science channels, 18 GHz rang) Corresponds to a state frame. • Scan: Observing interval within which target and frequency cycling pattern is unchanged

  7. Role of Digital Packaging Processor • To filter, average, partially calibrate and convert raw correlator output into a Miriad-compatible format that is written to Interim Data Base • Real time, irreversible processing

  8. DPP Interface Overview Interim Data Base DPP <P>, <P2>, Correlations Correlator Start / End Scan Commands Scan-independent Calibration Parameters Miriad format ACC Scan Parameters Frame parameters State Frame Frame status report Internal RFI Database RFI results 1 s timing tick 0.02 s timing tick

  9. DPP Task Timing • Occasional – non operational • Accept, store and preprocess calibration parameters • Scan initiation • Accept, store and preprocess scan-specific parameters • Data frame (20 ms) • filter, and frequency-average correlator output • Spectral frame (1 s) • Assemble, pre-calibrate, reformat and write data to Interim database • TBD • Format results and write to RFI database

  10. DPP – Stage 1 ProcessingEvery data frame (20ms) • Evaluate kurtosis data to identify RFI-affected subbands as a function of frequency only. • Save RFI statistics • Combine with pre-flagged subbands to generate a “destination vector” for each subband • Apply complex gains at subband level ??? • Average subband data into spectral channels • Save 1st 3 moments of averages ???

  11. DPP Stage 2 ProcessingEvery spectral frame (1s) • Convert antenna-based flags (e.g. slewing) from state frame to baseline-based, frequency-independent flags • Apply time-independent complex gains if available • Apply baseline corrections • Apply non-linearity corrections • Correct for attenuator settings • Correct for spectral simultaneity • Miriad format  this is no longer optional • Convert visibility, uv and analysis-relevant state-frame data to Miriad-compatible format • Write spectral frame to IDB • Report DPP status to state frame

  12. DPP - Implementation • Original concept was to follow FASR plan for a cluster-based DPP • Estimate processing requirements for EOVSA DPP at ~100 MIPS = 1/60 of FASR requirements • Implementation will be based on a single multi-core machine • Software organization will be compatible with migration to a cluster if necessary

  13. DPP Software Architecture RFI database IDB ACC State Frame Correlator DPP Coordination Task I/O, data assembly, no processing per se C1 Pointers within shared memory ParameterProcessing Header Processing Stage 1 Processing Stage 2 Processing C2 C2 C3, C4 C2 Conventional, time-independent processing tasks Cn = core within a quad core processor or nodes in a cluster

  14. DPP Status • Software architecture and tasks identified • Detailed definition of interfaces is underway • EOVSA to Miriad format conversion being tested with FST data • (Fortran 77 for Miriad compatibility) • Next: 1. Complete definition of interfaces 2. Code Stage 1 tasks (GH) • Evaluate timing requirements • Code Coordination task (JM). • Detailed definition of processing algorithms • Code of Stage 2 tasks 5. Machine selection and purchase • Development platform? • Goal: Functional DPP to support prototype testing

More Related