1 / 16

High Volume Data Processing

High Volume Data Processing. Discussion of NAVO’s PFM CRADA approach. David Fabre. Outline. Data Volumes Data Structures Old Way – GSF Swath editing New Way – PFM Area Based editing Near Future Way – CUBE PFM.

hazina
Download Presentation

High Volume Data Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. High Volume Data Processing Discussion of NAVO’s PFM CRADA approach David Fabre

  2. Outline • Data Volumes • Data Structures • Old Way – GSF Swath editing • New Way – PFM Area Based editing • Near Future Way – CUBE PFM

  3. Data Volumes – Deep Water(taken from L. Mayer of UNH – 29th MB Course, Lafayette, LA, 2002) In 3000 meters of water Method Soundings/hr mB/hr Lead Line 1 .000008 Echo Sounder 900 .0072 Multibeam 68,760 10.5 (Average with Backscatter)

  4. Data Volumes – Shallow Water(taken from L. Mayer of UNH – 29th MB Course, Lafayette, LA, 2002) In 100 meters of water Method Soundings/hr mB/hr Lead Line 10 .000008 Echo Sounder 21,600 .1728 EM-100 292,000 2.1 EM-1000 324,000 27.9 (bksctr) EM-3000 1,500,000 79.8 (bksctr) In 5 meters 17,000,000 450 (bksctr)

  5. Data Structures • Generic Sensor Format (GSF) • Time-based, primary key time • Pure File Magic (PFM) • Area-based, primary key area PFM Data Structure

  6. Old Way – Swath editing

  7. NAVO’s gsf_geoswath

  8. New Way – PFM Area Based editing

  9. SABER Multi-View Editor

  10. IVS Fledermaus Overview

  11. Fledermaus 3D Editor

  12. Maintaining the Links

  13. Near Future Way – CUBE PFM • We will edit were multiple possible surfaces are present (hypotheses) instead of “killing dots” • To be covered more thoroughly later by Lindsay Gee and Brian Calder

  14. Shallow Filtered and CUBE Best Estimate Overviews

  15. Editing Soundings vs. CUBE Hypothesized Surfaces

  16. Conclusion We’d like to make all products from a grid like the Navigation Surface (ex. DBDBV), which should attract different consumers of our data, and keep our GSF in archive for future re-use.

More Related