1 / 40

A “Short” Introduction to the IRIS Data Management Center Data Holdings Data Organization and Data Access

A “Short” Introduction to the IRIS Data Management Center Data Holdings Data Organization and Data Access. By Tim Ahern, Program Manager IRIS DMS Rick Benson, Director of Operations Philip Crotwell, Software Engineer Peg Johnson, USArray Data Analyst Joanna Muench, Software Engineer.

mikhail
Download Presentation

A “Short” Introduction to the IRIS Data Management Center Data Holdings Data Organization and Data Access

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A “Short” Introduction to the IRIS Data Management CenterData HoldingsData Organization andData Access By Tim Ahern, Program Manager IRIS DMS Rick Benson, Director of Operations Philip Crotwell, Software Engineer Peg Johnson, USArray Data Analyst Joanna Muench, Software Engineer

  2. By 5:00PM • You should have been exposed to many features of the DMC • You will not think this was a ‘short course’ • You will have lots of questions • But ask questions as we go along

  3. Agenda • Overview • Data Organization • Metadata • DMC Centered Request Mechanisms • Servicing Requests- the internal side • What users do when receiving data • FISSURES and the DHI • Distributed DHI Enabled Data Centers • DHI Clients • Using Real Time Data from the DMC • SPADE • Web Services - New Capabilities • Other Resources

  4. Overview: What is IRIS? • Consortium of 103 US Universities • National Science Foundation Funded • $12.5 - $14.5 million per year • Significant EarthScope Funding • Four Programs • GSN • 138 permanent stations • PASSCAL • 4000 portable channels • E&O • Workshops, posters, museum displays • DMS • 5000 stations over 40 years • Archives 9 terabytes per year • Distributes 12 terabyte per year • Services 200,000 data requests per year • Will serve about 1 billion seismograms

  5. IRIS HQ E&O PM The IRIS Programs Washington DC Data Generating Info Distribution Seattle Albuquerque San Diego GSN GSN PM, UofH DMS DMS PM Research Community Albuquerque San Diego PASSCAL PASSCAL PM E&O K-12 Public Socorro

  6. U. of S. Carolina ASL DCC Lamont WQC IDA DCC DMC KZNet Almaty UofW The IRIS Data Management System

  7. US Regional Networks USArray GSN PASSCAL FDSN DMC Foreign Networks PBO SAFOD Data Sources IRIS EarthScope

  8. More than 5,000 seismic stations have data available through the IRIS DMC

  9. 1126 Stations telemetered in real timefrom 54 different networks

  10. Data from 1966 -Present Temporary • Targets of Opportunity • 1966 - current • Unstructured Data • Apollo • Temporary Experiments • 1986 - current • Permanent Networks • 1970 - current Permanent Networks Targets of Opportunity

  11. Seismological Broadband Seismometers Strong Motion Buildings Structures Free Field Geophones Hydrophones Meteorological Wind Speed Wind Direction Temperature Humidity Rain Gauge Solar Radiaton Insolation/Pyranometer Microbarograph Relative/Absolute Infrasound microphones Types of Sensor Data IRIS manages • Geophysical • Magnetotelluric • Magnetometer • Electric Field • Strainmeter • dilatational • tensor • Tidal Pendulum • Gravimeter • conventional • superconducting • Tiltmeter • Creep Meters • Water Column • Water Current • Depth • Temperature • Water Level

  12. USArray Seismic Transportable Array Permanent Array Flexible Array Magnetotelluric Data SAFOD Seismic Data IRIS/DMC and NCEDC Monitoring Channels 250 samples/second Triggered Events 4000 samples/second Continuous Data 4000 samples/second PBO GPS Data Via UNAVCO Seismic Data Borehole Seismic Strain Borehole Strain Laser Strain EarthScope Data

  13. Naming Conventions • Active Source • SEGY or SEG2 and no naming convention • Passive Source • Majority of DMC Data Holdings • SEED Convention • Network Code - 2 characters • Station Code - 5 characters • Location Identifier - 2 characters • Channel Code - 3 characters • Band, Instrument, Orientation • Data Quality (DRQ) • Time • Virtual Networks Introduction

  14. 45 terabytes and growing

  15. End of Overview

  16. Data Request Tools at the IRIS DMC

  17. Request Methods

  18. breq_fast File Formathttp://www.iris.washington.edu/manuals/breq_fast.html .NAME Joe Seismologist .INST Podunk University .MAIL 101 Fast Lane, Middletown, KS 89432 .EMAIL joe@podunk.edu .PHONE 555 555-1212 .FAX 555 555-1213 .MEDIA DAT .ALTERNATE MEDIA 1/2" tape - 6250 .ALTERNATE MEDIA EXABYTE .LABEL Joe's FIRST Request .SOURCE ~NEIC PDE~Jan 1990 PDE~National Earthquake Information Center - USGS DOI~ .HYPO ~1990 01 02 20 21 32.62~ 13.408~ 144.439~135.0~18~216~Mariana Islands~ .MAGNITUDE ~5.7~mb~ .QUALITY [B,E,D,R,Q] .END ANTO IU 1995 01 02 02 10 36.6 1995 01 02 02 12 36.6 1 SH? 00 GRFO IU 1995 01 02 02 10 37.1 1995 01 02 02 12 37.1 1 SH? 10 SEE CD 1995 01 02 14 45 08.9 1995 01 02 14 47 08.9 1 SHZ NNA II 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ PFO TS 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ 10 KMI CD 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ 00 SSE CD 1995 01 04 02 18 25.4 1995 01 04 02 20 25.4 2 B?? SHZ 00 PAS TS 1995 1 4 2 10 49 1995 1 4 2 12 49 3 BH? SHZ L?? 00 Left justified and .END

  19. Email Request to • breq_fast@iris.washington.edu • DATALESS@iris.washington.edu • Miniseed@iris.washington.edu • sync@iris.washington.edu

  20. Better Ways to Build breq_fast files • WebRequest http://www.iris.edu/data/WebRequest.htm

  21. AutoDRM Request Formathttp://seismo.ethz.ch/autodrm/ BEGIN GSE2.1 Begin of message MSG_TYPE request This is a request MSG_ID your_specific_ID DEU_NDC Your ID TIME 1994/12/18 22:54:4.0 TO 1994/12/18 22:56:12.0 Define Start/End Time STA_LIST APL Define station(s) CHAN_LIST SHZ, SLZ Define channel(s) WAVEFORM GSE2.1 You want waveforms STATION GSE2.1 Station-info needed CHANNEL GSE2.1 Channel-info needed RESPONSE GSE2.1 Send calibration-info OUTAGE GSE2.1 Send outage info E-MAIL fish@sdac.bgr.hannover.de Your e-mail address STOP End of request

  22. Distributed Data Center Technologies

  23. CNSN Canada NetDC Request NCEDC USA SCEDC USA FDSN.ORG IRIS DMC USA GEOSCOPE France PACIFIC 21 Japan ORFEUS Netherlands Networking Data Centers by Email • Routes user requests between data centers • Wraps normal Data Center operations within an application layer • Coordinates delivery of information back to a user • Peer based system • Any NetDC node can act as a hub data center • Or FDSN.ORG can be used MedNet Italy

  24. An Example NetDC Request • .NETDC_REQUEST • .NAME Tim Ahern • .INST IRIS DMC • .EMAIL tim@iris.washington.edu • .PHONE 206 547-0393 • .FAX 206 547-1093 • .LABEL Example for Training Course • .MEDIA FTP • .ALTERNATE MEDIA DAT • .FORMAT_WAVEFORM SEED • .FORMAT_RESPONSE SEED_ASCII • .MERGE_DATA NO • .DISPOSITION PULL • .END • .INV * IC • .DATA * IC * ?? "*" "2003 01 01 00 00 00.0" "2003 01 01 00 10 00.0"

  25. Types of NetDC Information • .TOKEN DataCenter Network Station Location Channel Starttime Endtime • Inventory (.INV) • .INV *
 • .INV GEOFON *
 • .INV GEOFON AA B*
 • .INV GEOFON AA TATO * *
 • .INV * IU "AAE ANMO" * * "1995 03 03 02 24 01.3" "1995 03 03 07 00 30.0"
 • .INV * II KIV * "BHE BHN BHZ" "1996 05 01 00 00 00.0" "1996 05 01 05 00 00.0" • Responses (.RESP) • .RESP * G SSBC * * "1990 03 01 00 00 00" "1990 03 02 00 00 00" • Waveforms (.DATA) • .DATA * II BFO * BHZ "1998 02 03 04 20 34" "1998 03 04 20 20 00"

  26. NetDC Request Formhttp://www.iris.edu/data/netdcForm.htm

  27. Example NetDC Output .INV IRIS_DMC IC * [AVAILABLE NETWORKS][NET_CODE NETWORK_NAME OPERATORS COMMENTS] "IC" "IRIS China Digital Seismic Network" "USGS Albuquerque Seismological Laboratory" ” ” [AVAILABLE STATIONS][STATION LATITUDE LONGITUDE ELEVATION DESCRIPTION START_EFF_TIME END_EFF_TIME] "BJT" "40.0183" "116.1679" "197" "Baijiatuan, Beijing, China" "1994,145,00:00:00" "ENH" "30.2718" "109.4868" "487" "Enshi, Hubei Province, China" "1997,263,00:00:00" "HIA" "49.2667" "119.7417" "610" "Hailar, Neimenggu Province, China" "1994,338,00:00:00" "KMI" "25.1233" "102.74" "1975" "Kunming, Yunnan Province, China" "1996,118,00:00:00" "LSA" "29.7" "91.15" "3789" "Tibet, China" "1995,215,00:00:00" "2599,365,23:59:59” "MDJ" "44.6164" "129.5919" "250" "Mudanjiang, Heilongjiang Province, China" "1996,314,00:00:00" "QIZ" "19.0294" "109.8433" "230" "Qiongzhong, Guangduong Province, China" "2001,049,00:00:00" "SSE" "31.0951" "121.1859" "15" "Shanghai, China" "1996,263,00:00:00" "2599,365,23:59:59” "WMQ" "43.8211" "87.695" "903" "Urumqi, Xinjiang Province, China" "1995,266,00:00:00" "XAN" "34.0313" "108.9237" "630" "Xi'an, China" "1992,334,00:00:00"

  28. Submitting NetDC Requests mail netdc@fdsn.org

  29. Distributed Data Center Technologies FISSURES and the Data Handling Interface

  30. FISSURES • 1998 FISSURES Workshop • Industry Leaders (OpenGIS, Visio, etc) • Move to object oriented standards based systems • Community recommends developing CORBA-based Data Access • 1999 - 2000 • Interface Definition Language (IDL) • Adult Supervision by 2AB • Lead by University of South Carolina • 2000-2001 • FISSURES downsized to Data Handling Interface (DHI) • Servers developed by IRIS DMC • Initial Clients developed by USC, IRIS, ISTI • 2002-2004 • Server refinement and hardening • Client Development (also SNL, LLNL, U. of Washington) • Server propagation to other data centers (Berkeley) • 2005 • Web Services extension • International Expansion (Caltech, ORFEUS, GEOFON, MEDNET) • GEOSS • 2006 • Continued Expansion (ORFEUS, JAMSTEC, ISC, NEIC?)

  31. DHI Data Center DHI Data Center IDL Network Network IDL Waveforms Waveforms IDL Events Events DHI Data Center DHI Data Center Network Network Waveforms Waveforms Events Events Data Handling Interface (DHI)Access to Distributed Data Centers DHI Clients VASE SOD jWEED etc.

  32. Distributed Data Center Technologies DHI Enabled Data Centers

  33. DHI Enabled Centers Existing Planned Planned (Events)

  34. DHI Enabled Centers- European Existing Planned Planned (Events)

  35. Using Real Time Data from the IRIS DMC

  36. BB Array BK AT TA NN PN LD AZ IU H2 IC IM II AK EQ NP FA LISS (3) CI Antelope ORB (24) GT CD-1 AU KZ TW MY KN CZ BUD Buffer of Uniform Data SC AR XX MB SEEDLink (5) Earthworm Waveserver (21) UU CH NL UW GE MN NI PBO UO PR PA WY NM IW ER ER RE CC PE ET IE LI US LB Real Time Data at IRIS

  37. LISS SeedLink DHI BUD Buffer of Uniform Data Real Time Data Flow from DMC Antelope Earthworm SeisGram2K VASE SOD EMM

  38. More information - BUD Page

  39. SEEDlink Usage --- for example

  40. LISS Usage --- for example

More Related