1 / 56

Evolution of the SZ-2 Algorithm

Evolution of the SZ-2 Algorithm. Sebastian Torres CIMMS/NSSL Technical Interchange Meeting Fall 2006. Recent Evolution of SZ-2. Several “areas of interest” identified and ranked by the ROC Solutions to critical AOIs were discussed at previous TIM Assessment of engineering fixes

neorah
Download Presentation

Evolution of the SZ-2 Algorithm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evolution of the SZ-2 Algorithm Sebastian Torres CIMMS/NSSL Technical Interchange Meeting Fall 2006

  2. Recent Evolution of SZ-2 • Several “areas of interest” identified and ranked by the ROC • Solutions to critical AOIs were discussed at previous TIM • Assessment of engineering fixes • After-TIM discussions • Provided draft AEL • Analyzed ROC’s implementation • Last AEL delivered on June 9, 2006 • Real-time implementation validated with off-line MATLAB simulator • SZ-2 implemented following latest AEL • Many cases presented to the DQ team • Problem with noisy velocities due to bug in the code • TAC approved SZ-2 for Build 9 of ORDA

  3. 06/09/06 AEL Revisions • Algorithm description fits real-time implementation • dB-for-dB censoring • Strong-point clutter suppression • Required outputs (T0, R0, R1, R2) • Addition of overall logic flow • Efficient processing of non-overlaid echoes • Dynamic use of data windows • Spectrum width computations • Unbiased autocorrelation for data windows • Adaptable spectrum width estimator • Censoring • dB-for-dB • Rules and classification • Strong-point clutter suppression

  4. Dynamic Use of Data Windows • SZ-2 uses three data windows depending on the situation • The PNF needs the von Hann (or more aggressive) window • GMAP needs the Blackman window to achieve required clutter suppression • Dynamic data windowing rules • Use the rectangular window with non-overlaid, non-clutter-contaminated echoes • Implemented algorithm uses the “default” window (currently the Hamming window) • Use the von Hann window with overlaid, non-clutter-contaminated echoes • Use the Blackman window with clutter-contaminated echoes

  5. Divide by the number of pairs? Correct factor for arbitrary data windows Spectrum Width Computations • Rules for choosing a spectrum width estimator • Use the R0/R1 estimator with non-overlaid echoes • Use the R1/R2 estimator with overlaid echoes • Unbiased spectrum width estimates require unbiased autocorrelation estimates • The data window must be accounted for in the autocorrelation estimator

  6. Censoring in SZ-2 • What is “black” and what is “purple”? • Aimed at clear classification rules • “Black” corresponds to non-significant returns • “Purple” corresponds to gates that have significant returns but cannot be recovered • Need to maintain accepted system behavior • For example, dB-for-dB censoring is tagged as “black” • Gates are classified as having one of the following three types of returns • Signal return: above adjusted SNR threshold and recoverable (passes all tests) • Noise return: below SNR threshold or strong clutter in non-overlaid case • Overlaid return: Unrecoverable with two or more overlaid trips (at least fails one of the tests)

  7. Censoring Rules Strong Trip Censoring Rules

  8. Censoring Rules Weak Trip Censoring Rules

  9. Censoring Rules Other Trip* Censoring Rules * This censoring applies to the two weakest trips

  10. Data Windowing Issues Sebastian Torres CIMMS/NSSL Technical Interchange Meeting Fall 2006

  11. Data windows reduce the equivalent number of independent samples available to estimate spectral moments Non-rectangular windows are tapered so end samples “contribute less” to the estimation process The more aggressive the data window, the larger the errors of estimates for all spectral moments Data windows need to be accounted for in the autocorrelation estimator Normalization by “lag window” (D&Z, ch5) The Effect of Data Windows

  12. Standard Error of Moment Estimates for Different Windows vs. SNR Parameters of VCP 211 M = 64, PRT = 780 ms (PRI #8), f = 2800 MHz, sv = 4 m/s

  13. Relative Standard Error of Moment Estimates for Different Windows vs. SNR Parameters of VCP 211 M = 64, PRT = 780 ms (PRI #8), f = 2800 MHz, sv = 4 m/s

  14. Default window with non-overlaid, non-clutter-contaminated echoes AEL recommends using the rectangular window ORDA uses the Hamming window as the default window von Hann with overlaid, non-clutter-contaminated echoes Blackman with clutter-contaminated echoes Data Windows in SZ-2 Parameters of VCP 211 M = 64, PRT = 780 ms (PRI #8), f = 2800 MHzTrue sv = 4 m/s, SNR for Z & sv = 10 dB, SNR for v = 8 dB

  15. SZ-2 VelocityHamming Window KCRI (ORDA)VCP 211 - 03/19/06

  16. SZ-2 VelocityRectangular Window KCRI (ORDA)VCP 211 - 03/19/06

  17. SZ-2 and Super Resolution(NPI) Sebastian Torres CIMMS/NSSL Technical Interchange Meeting Fall 2006

  18. Legacy Resolution Super Resolution Two elements of super-resolution data What is Super Resolution? • Legacy Resolution spatial sampling • Reflectivity: 1-km by 1-deg grid • Doppler: 250-m by 1-deg grid • Super Resolution spatial sampling • All moments: 250-m by 0.5-deg grid • Finer spatial sampling and smaller resolution volume lead to about 50% improvement in range of detection for mesocyclone and tornado signatures (Brown et al. 2002) Tornado outbreak in Oklahoma City9 May 2003 (Curtis et al. 2003)

  19. Elevation Azimuth Super resolution base data Super Resolution for NEXRAD • Super-resolution data scheduled for operational use on NEXRAD • Short-term goals - Phase I: ORDA Build 10 (FY 2008) • Data used for visualization only • Legacy- and super-resolution data available in the RPG • RPG algorithms ingest legacy-resolution data • Long-term goals - Phase II: ORDA Build 13? (FY 2012) • Data used by the algorithms • Super-resolution data produced on lower-elevation scans (split cuts) • Higher likelihood of finding tornado and meso signatures • SZ-2 may also run on these scans Z RDA RDA RPG v Products w

  20. 1 deg 0.5 deg 250 m 1 km Super Resolution on the ORDA (I) • RDA must produce base data with finer spatial sampling and resolution • Finer spatial sampling grid • Radials collected at 0.5 deg azimuth intervals • Legacy-resolution radials collected at: 0.5, 1.5, 2.5, … deg • Super-resolution radials collected at: 0.25, 0.75, 1.25, … deg • Recombined radials created at: 0.5, 1.5, 2.5, … deg • No range averaging to maintain 250 m sampling in range • Finer resolution • Selective data windowing Legacy-resolutionreflectivity grid Super-resolutionreflectivity grid

  21. M = 64 1 deg Pulse # 0 deg 0.5 deg 1.0 deg 1.5 deg 2.0 deg GCF Map bypass bypass bypass filter bypass Super Resolution on the ORDA (II) • Solution: Overlapping 1-deg radials with data windowing sampled every 0.5 deg and no range averaging • For each range gate, M time-series data samples are weighted • … with von Hann window if clutter filtering is not needed • … with Blackman window if clutter filtering is needed

  22. Super Resolution on the ORDA (III) • Doppler-derived reflectivities and noise power are needed in the ORPG to produce legacy-like data • Reflectivity is range unfolded together with the Doppler moments • Noise power is added to metadata • Doppler moments produced up to 300 km • Data beyond 230 km is not discarded • Throughput is doubled • Twice the number of radials in an elevation cut • Computational complexity is doubled • Super-resolution radials have the same number of samples as legacy-resolution radials

  23. Doppler velocity v Radial 1PSD Radial 2PSD Acceptabledata quality v1 v2 Super Resolution on the ORPG (I) • ORPG algorithms expect data with legacy resolution and quality • Super-resolution data does not have the required resolution or quality for the algorithms (it’s OK for visualization) • Super resolution radial recombination • Two super-resolution radials are recombined to form one legacy-resolution radial • Recombination assumes a bimodal spectrum • Approach exhibitslow risk and provides data with acceptable quality • Algorithm must deal with missing data • SNR thresholds • Overlaid echoes

  24. Super Resolution on the ORPG (II) • Velocity dealiasing algorithm must run on super-resolution and recombined legacy-resolution data • Both velocity fields will available for visualization • Legacy-resolution fields will be fed to the algorithms • Increased processing requirements

  25. SZ-2 Changes to run inSuper Resolution mode • ORDA • Finer spatial sampling grid: no changes • Finer resolution: no changes • Default window for super resolution will be set to von Hann • Range unfolding of Doppler reflectivities: no changes • Unfolded Doppler powers already exist within SZ-2 • Addition of noise power to metadata: no changes • Doppler data up to 300 km: no changes • Data already exist within SZ-2 • Doubled throughput: no changes • Doubled CPU load: needs testing • Current maximum CPU usage with SZ-2 is ~35% • Expect maximum CPU usage of ~70% with super-resolution SZ-2 • ORPG • SZ-2 is already transparent to the ORPG: no changes

  26. Reflectivity – FFTLegacy resolution

  27. Velocity – FFTLegacy resolution

  28. Reflectivity – SZ-2Legacy resolution

  29. Reflectivity – FFTLegacy resolution

  30. Velocity – SZ-2Legacy resolution

  31. Velocity – FFTLegacy resolution

  32. Reflectivity – SZ-2Super resolution

  33. Reflectivity – SZ-2Legacy resolution (Recombined)

  34. Reflectivity – SZ-2Legacy resolution

  35. Velocity – SZ-2Super resolution

  36. Velocity – SZ-2Legacy resolution (Recombined)

  37. Velocity – SZ-2Legacy resolution

  38. “All-Bins” Clutter Filtering and SZ-2 Sebastian Torres CIMMS/NSSL Technical Interchange Meeting Fall 2006

  39. “All-Bins” Clutter Filteringand SZ-2 • SZ-2 cannot recover overlaid signals if multiple trips have clutter contamination (overlaid clutter) • Operator-selected “all-bins” clutter filtering forces overlaid clutter in every bin • A large percentage of the bins will not have clutter contamination • Is there a simple way to detect which bins have clutter contamination? • Answer: GMAP

  40. “All-Bins” Clutter Filteringand SZ-2 • GMAP is used to “detect” clutter • The clutter power removed by GMAP during the long PRT is used as an indicator of the presence of clutter • Recommended SZ-2 algorithm uses the long-PRT CSR • Preliminary tests show that the long-PRT CNR may be a better indicator • We rely on only one estimate from GMAP

  41. Reflectivity – All bins KCRI (ORDA)VCP 211 - 03/19/06

  42. Reflectivity – Bypass Map KCRI (ORDA)VCP 211 - 03/19/06

  43. Velocity – All bins GCF Clutter if CSR > 15 dB KCRI (ORDA)VCP 211 - 03/19/06

  44. Velocity – Bypass Map KCRI (ORDA)VCP 211 - 03/19/06

  45. Velocity – All bins GCFClutter if CSR > 10 dB KCRI (ORDA)VCP 211 - 03/19/06

  46. Velocity – All bins GCF Clutter if CSR > 15 dB KCRI (ORDA)VCP 211 - 03/19/06

  47. Censoring – All bins GCFClutter if CSR > 10 dB KCRI (ORDA)VCP 211 - 03/19/06

  48. Velocity – All bins GCFClutter if CNR > 10 dB KCRI (ORDA)VCP 211 - 03/19/06

  49. Censoring – All bins GCFClutter if CNR > 10 dB KCRI (ORDA)VCP 211 - 03/19/06

  50. Velocity – All bins GCFClutter if CNR > 20 dB KCRI (ORDA)VCP 211 - 03/19/06

More Related