1 / 20

Data centre support for the IGS-RT PP

Data centre support for the IGS-RT PP. W. Söhne, H. Habrich, G. Weber Federal Agency for Cartography and Geodesy , Frankfurt am Main, Germany. Outline. Introduction IGS-RT PP Data and product file centre File types under consideration Technical aspects Data volume Up- and downloading

lani
Download Presentation

Data centre support for the IGS-RT PP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data centre support for the IGS-RT PP W. Söhne, H. Habrich, G. Weber Federal Agency for Cartography and Geodesy, Frankfurt am Main, Germany

  2. Outline • Introduction • IGS-RT PP • Data and product file centre • File types under consideration • Technical aspects • Data volume • Up- and downloading • Completeness • Limitations • Policy aspects • Upload policy • Conclusions & open questions

  3. Introduction • Storing of highrate files nothing new for IGS • At CDDIS back to 2001, doy121 • Initiative from the LEO needs • At IGN • Others • Term “highrate” is a relative one • Usually means 1 Hz sampling rate • But higher sampling rates in use depending on the application • Currently ~110 stations at CDDIS • Clear statement from IGS concerning the needs of long-term archive still missing

  4. IGS-RT PP (1) • One activity of the IGS-RT PP: Data and product file centre • File types under consideration • Navigational data (broadcast ephemeris) • Observational data • highrate RINEX files • hourly and daily RINEX files • Derived files (products) • Orbits or orbit corrections • Clocks or clock corrections • Troposphere & ionosphere parameters

  5. IGS-RT PP (2) • Why generation of highrate files? • Currently, considerable number of users are “feeding” near real-time applications • For future scientific analyses • What is “near real-time”? • Typically hourly applications, with 20-30 minutes computation time •  two times per hour computation possible • Why 15 minutes files? • Reduction of file size compared to an hourly highrate file • “Convention”

  6. IGS-RT PP (3) • Is that interval small enough? Are there other intervals of interest, e.g. 5 minutes files? • Advantages • Better distribution of uploading over time • Other intervals can easily be created/derived • File naming convention fits • Disadvantages • Interval size not known in IGS • Number of files growing

  7. Technical aspects: data volume • Data volume of highrate observational files • Example BUTE115A45.08O: 1 Hz 15 minutes GPS+GLONASS RINEX file, # obs. types 8, 20 SV in view: 2.4 Mb • Hatanaka+compress: 170 Kb • 5.7 Gb per station and year for the compressed files •  >> 500 Gb per year • Selection of IGS stations for high-rate storing? • Data volume of product files • Clocks: 1 Hz, 1 hour: 2.5 Mb, compressed 170 Kb5.7 Gb per solution and year • …

  8. Technical aspects: up- and downloading • Data upload of observational files • Example 250 stations, 24 files per day (hourly): 6000 files uploaded * 100 Kb  0.6 Gb per day • Example 110 stations, 96 files per day (highrate): 10560 files uploaded * 170 Kb  1.7 Gb per day • Critical aspects are • Is the data centre able to handle all incoming files within, e.g., 5 minutes after the full hour?If not  spreading of upload over a certain time span?! • Is the data centre able to handle the parallel download requests in peak periods?

  9. Technical aspects: completeness (1) • “Completeness” covers the aspects • Number of observation types • Number of epochs • Number of SVs per epoch • Ratio observed / predicted number of observations • Completeness can be affected by • Outages of single stations data streams • Outages of the broadcasting system • Handling of unhealthy SVs • Limitation of supported observation types

  10. Technical aspects: limitations (1) • “Limitations” cover the following aspects • Resolution of • Phase • Code • Support of • GPS L2C, L5 • GLONASS • Galileo • SBAS

  11. Technical aspects: completeness (2) • Different levels of comparison of original and accumulated files possible • Character by character • Difficult due to roll-over phase values • Completeness • Epochs missing? • SVs missing? • Parallel analysis • Differences between the results

  12. Technical aspects: completeness (3) Hourly file BUTE115A.08O (TEQC) Highrate file BUTE115A00.08O (BNC)

  13. Technical aspects: limitations (2) ALME: file from TEQC vs. file from BNC (RTCM 2.3) after RNXSMT (BSW5.0)

  14. Technical aspects: limitations (3) WTZR: file from TEQC vs. file from BNC (RTCM 3.0) after RNXSMT (BSW5.0)

  15. 0 2 0 1 6 0 1 2 1 56 0 1 35 2 100,0% 100,0% 100,0% 1 0 0 58 2 99,9 -100,0% 99,9 -100,0% 99,9 -100,0% 4 8 99,8 - 99,9% 99,8 - 99,9% 99,8 - 99,9% 21 99,7 - 99,8% 99,7 - 99,8% 99,7 - 99,8% 99,6 - 99,7% 99,6 - 99,7% 99,6 - 99,7% 99,5 - 99,6% 99,5 - 99,6% 99,5 - 99,6% 0 99,0 - 99,5% 99,0 - 99,5% 99,0 - 99,5% 90,0 - 99,0% 90,0 - 99,0% 90,0 - 99,0% 1 50,0 - 90,0% 50,0 - 90,0% 50,0 - 90,0% 168 0 579 14 < 50,0% < 50,0% < 50,0% 0 5 Technical aspects: completeness (4) Completeness of 711 hourly high-rate RINEX files from RTCMv3 streams coming in via NTRIP/TCP Completeness of 233 hourly high-rate RINEX files from RTCMv2 streams coming in via NTRIP/TCP Completeness of 24 hourly high-rate RINEX files from RTIGS streams coming in via udpRelay

  16. Streaming interrupted Missing SVs G06, G21, G26, G30 Longer sections w/o GLONASS Technical aspects: completeness (5)

  17. Policy aspects (1) „Traditional“ situation transferring hourly files, daily files, ephemeris files Station A Data centre 1 User a Station B User b Mirroring Station C Data centre 2 User c Mirroring Data centre 3 Station N User m  file creation at the station

  18. transferring files transferring streams Policy aspects (2) Possible situation using real-time data streams for RINEX file creation Broadcaster 1 Station A User a Station B or Station C User b Data centre 1 Station N  file creation possible at the data centre, at the users site, at a third party

  19. Conclusions & open questions (1) • Derivation of highrate RINEX files from real-time streams suitable tool • Current interval for highrate files 15 minutes – is that small enough? •  Recommendation #1: Storing of 1 Hz 15 minutes files, at least for the long-term archive • Derivation of daily files from streams instead of ftp transfer, at least for new or proposed stations? • RT-IGS PP CfP found five candidates (BKG, CDDIS, GA, KASI, Univ. of Padova)

  20. Conclusions & open questions (2) • Limitations acceptable? If yes, how to point the users to these limitations effectively? • Derivation of RINEX files from real-time streams possible for everyone – needs some regulation or clarification: Who is allowed to upload files derived from real-time data streams? •  Recommendation #2: highrate file creation and upload to the GDC in one hand, ideally at the broadcaster’s side

More Related