410 likes | 667 Views
A “Short” Introduction to the IRIS Data Management Center Data Holdings Data Organization and Data Access. By Tim Ahern, Program Manager IRIS DMS Rick Benson, Director of Operations Philip Crotwell, Software Engineer Peg Johnson, USArray Data Analyst Joanna Muench, Software Engineer.
E N D
A “Short” Introduction to the IRIS Data Management CenterData HoldingsData Organization andData Access By Tim Ahern, Program Manager IRIS DMS Rick Benson, Director of Operations Philip Crotwell, Software Engineer Peg Johnson, USArray Data Analyst Joanna Muench, Software Engineer
By 5:00PM • You should have been exposed to many features of the DMC • You will not think this was a ‘short course’ • You will have lots of questions • But ask questions as we go along
Agenda • Overview • Data Organization • Metadata • DMC Centered Request Mechanisms • Servicing Requests- the internal side • What users do when receiving data • FISSURES and the DHI • Distributed DHI Enabled Data Centers • DHI Clients • Using Real Time Data from the DMC • SPADE • Web Services - New Capabilities • Other Resources
Overview: What is IRIS? • Consortium of 103 US Universities • National Science Foundation Funded • $12.5 - $14.5 million per year • Significant EarthScope Funding • Four Programs • GSN • 138 permanent stations • PASSCAL • 4000 portable channels • E&O • Workshops, posters, museum displays • DMS • 5000 stations over 40 years • Archives 9 terabytes per year • Distributes 12 terabyte per year • Services 200,000 data requests per year • Will serve about 1 billion seismograms
IRIS HQ E&O PM The IRIS Programs Washington DC Data Generating Info Distribution Seattle Albuquerque San Diego GSN GSN PM, UofH DMS DMS PM Research Community Albuquerque San Diego PASSCAL PASSCAL PM E&O K-12 Public Socorro
U. of S. Carolina ASL DCC Lamont WQC IDA DCC DMC KZNet Almaty UofW The IRIS Data Management System
US Regional Networks USArray GSN PASSCAL FDSN DMC Foreign Networks PBO SAFOD Data Sources IRIS EarthScope
More than 5,000 seismic stations have data available through the IRIS DMC
1126 Stations telemetered in real timefrom 54 different networks
Data from 1966 -Present Temporary • Targets of Opportunity • 1966 - current • Unstructured Data • Apollo • Temporary Experiments • 1986 - current • Permanent Networks • 1970 - current Permanent Networks Targets of Opportunity
Seismological Broadband Seismometers Strong Motion Buildings Structures Free Field Geophones Hydrophones Meteorological Wind Speed Wind Direction Temperature Humidity Rain Gauge Solar Radiaton Insolation/Pyranometer Microbarograph Relative/Absolute Infrasound microphones Types of Sensor Data IRIS manages • Geophysical • Magnetotelluric • Magnetometer • Electric Field • Strainmeter • dilatational • tensor • Tidal Pendulum • Gravimeter • conventional • superconducting • Tiltmeter • Creep Meters • Water Column • Water Current • Depth • Temperature • Water Level
USArray Seismic Transportable Array Permanent Array Flexible Array Magnetotelluric Data SAFOD Seismic Data IRIS/DMC and NCEDC Monitoring Channels 250 samples/second Triggered Events 4000 samples/second Continuous Data 4000 samples/second PBO GPS Data Via UNAVCO Seismic Data Borehole Seismic Strain Borehole Strain Laser Strain EarthScope Data
Naming Conventions • Active Source • SEGY or SEG2 and no naming convention • Passive Source • Majority of DMC Data Holdings • SEED Convention • Network Code - 2 characters • Station Code - 5 characters • Location Identifier - 2 characters • Channel Code - 3 characters • Band, Instrument, Orientation • Data Quality (DRQ) • Time • Virtual Networks Introduction
breq_fast File Formathttp://www.iris.washington.edu/manuals/breq_fast.html .NAME Joe Seismologist .INST Podunk University .MAIL 101 Fast Lane, Middletown, KS 89432 .EMAIL joe@podunk.edu .PHONE 555 555-1212 .FAX 555 555-1213 .MEDIA DAT .ALTERNATE MEDIA 1/2" tape - 6250 .ALTERNATE MEDIA EXABYTE .LABEL Joe's FIRST Request .SOURCE ~NEIC PDE~Jan 1990 PDE~National Earthquake Information Center - USGS DOI~ .HYPO ~1990 01 02 20 21 32.62~ 13.408~ 144.439~135.0~18~216~Mariana Islands~ .MAGNITUDE ~5.7~mb~ .QUALITY [B,E,D,R,Q] .END ANTO IU 1995 01 02 02 10 36.6 1995 01 02 02 12 36.6 1 SH? 00 GRFO IU 1995 01 02 02 10 37.1 1995 01 02 02 12 37.1 1 SH? 10 SEE CD 1995 01 02 14 45 08.9 1995 01 02 14 47 08.9 1 SHZ NNA II 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ PFO TS 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ 10 KMI CD 1995 01 04 02 41 57.5 1995 01 04 02 43 57.5 1 BHZ 00 SSE CD 1995 01 04 02 18 25.4 1995 01 04 02 20 25.4 2 B?? SHZ 00 PAS TS 1995 1 4 2 10 49 1995 1 4 2 12 49 3 BH? SHZ L?? 00 Left justified and .END
Email Request to • breq_fast@iris.washington.edu • DATALESS@iris.washington.edu • Miniseed@iris.washington.edu • sync@iris.washington.edu
Better Ways to Build breq_fast files • WebRequest http://www.iris.edu/data/WebRequest.htm
AutoDRM Request Formathttp://seismo.ethz.ch/autodrm/ BEGIN GSE2.1 Begin of message MSG_TYPE request This is a request MSG_ID your_specific_ID DEU_NDC Your ID TIME 1994/12/18 22:54:4.0 TO 1994/12/18 22:56:12.0 Define Start/End Time STA_LIST APL Define station(s) CHAN_LIST SHZ, SLZ Define channel(s) WAVEFORM GSE2.1 You want waveforms STATION GSE2.1 Station-info needed CHANNEL GSE2.1 Channel-info needed RESPONSE GSE2.1 Send calibration-info OUTAGE GSE2.1 Send outage info E-MAIL fish@sdac.bgr.hannover.de Your e-mail address STOP End of request
CNSN Canada NetDC Request NCEDC USA SCEDC USA FDSN.ORG IRIS DMC USA GEOSCOPE France PACIFIC 21 Japan ORFEUS Netherlands Networking Data Centers by Email • Routes user requests between data centers • Wraps normal Data Center operations within an application layer • Coordinates delivery of information back to a user • Peer based system • Any NetDC node can act as a hub data center • Or FDSN.ORG can be used MedNet Italy
An Example NetDC Request • .NETDC_REQUEST • .NAME Tim Ahern • .INST IRIS DMC • .EMAIL tim@iris.washington.edu • .PHONE 206 547-0393 • .FAX 206 547-1093 • .LABEL Example for Training Course • .MEDIA FTP • .ALTERNATE MEDIA DAT • .FORMAT_WAVEFORM SEED • .FORMAT_RESPONSE SEED_ASCII • .MERGE_DATA NO • .DISPOSITION PULL • .END • .INV * IC • .DATA * IC * ?? "*" "2003 01 01 00 00 00.0" "2003 01 01 00 10 00.0"
Types of NetDC Information • .TOKEN DataCenter Network Station Location Channel Starttime Endtime • Inventory (.INV) • .INV * • .INV GEOFON * • .INV GEOFON AA B* • .INV GEOFON AA TATO * * • .INV * IU "AAE ANMO" * * "1995 03 03 02 24 01.3" "1995 03 03 07 00 30.0" • .INV * II KIV * "BHE BHN BHZ" "1996 05 01 00 00 00.0" "1996 05 01 05 00 00.0" • Responses (.RESP) • .RESP * G SSBC * * "1990 03 01 00 00 00" "1990 03 02 00 00 00" • Waveforms (.DATA) • .DATA * II BFO * BHZ "1998 02 03 04 20 34" "1998 03 04 20 20 00"
Example NetDC Output .INV IRIS_DMC IC * [AVAILABLE NETWORKS][NET_CODE NETWORK_NAME OPERATORS COMMENTS] "IC" "IRIS China Digital Seismic Network" "USGS Albuquerque Seismological Laboratory" ” ” [AVAILABLE STATIONS][STATION LATITUDE LONGITUDE ELEVATION DESCRIPTION START_EFF_TIME END_EFF_TIME] "BJT" "40.0183" "116.1679" "197" "Baijiatuan, Beijing, China" "1994,145,00:00:00" "ENH" "30.2718" "109.4868" "487" "Enshi, Hubei Province, China" "1997,263,00:00:00" "HIA" "49.2667" "119.7417" "610" "Hailar, Neimenggu Province, China" "1994,338,00:00:00" "KMI" "25.1233" "102.74" "1975" "Kunming, Yunnan Province, China" "1996,118,00:00:00" "LSA" "29.7" "91.15" "3789" "Tibet, China" "1995,215,00:00:00" "2599,365,23:59:59” "MDJ" "44.6164" "129.5919" "250" "Mudanjiang, Heilongjiang Province, China" "1996,314,00:00:00" "QIZ" "19.0294" "109.8433" "230" "Qiongzhong, Guangduong Province, China" "2001,049,00:00:00" "SSE" "31.0951" "121.1859" "15" "Shanghai, China" "1996,263,00:00:00" "2599,365,23:59:59” "WMQ" "43.8211" "87.695" "903" "Urumqi, Xinjiang Province, China" "1995,266,00:00:00" "XAN" "34.0313" "108.9237" "630" "Xi'an, China" "1992,334,00:00:00"
Submitting NetDC Requests mail netdc@fdsn.org
Distributed Data Center Technologies FISSURES and the Data Handling Interface
FISSURES • 1998 FISSURES Workshop • Industry Leaders (OpenGIS, Visio, etc) • Move to object oriented standards based systems • Community recommends developing CORBA-based Data Access • 1999 - 2000 • Interface Definition Language (IDL) • Adult Supervision by 2AB • Lead by University of South Carolina • 2000-2001 • FISSURES downsized to Data Handling Interface (DHI) • Servers developed by IRIS DMC • Initial Clients developed by USC, IRIS, ISTI • 2002-2004 • Server refinement and hardening • Client Development (also SNL, LLNL, U. of Washington) • Server propagation to other data centers (Berkeley) • 2005 • Web Services extension • International Expansion (Caltech, ORFEUS, GEOFON, MEDNET) • GEOSS • 2006 • Continued Expansion (ORFEUS, JAMSTEC, ISC, NEIC?)
DHI Data Center DHI Data Center IDL Network Network IDL Waveforms Waveforms IDL Events Events DHI Data Center DHI Data Center Network Network Waveforms Waveforms Events Events Data Handling Interface (DHI)Access to Distributed Data Centers DHI Clients VASE SOD jWEED etc.
Distributed Data Center Technologies DHI Enabled Data Centers
DHI Enabled Centers Existing Planned Planned (Events)
DHI Enabled Centers- European Existing Planned Planned (Events)
Using Real Time Data from the IRIS DMC
BB Array BK AT TA NN PN LD AZ IU H2 IC IM II AK EQ NP FA LISS (3) CI Antelope ORB (24) GT CD-1 AU KZ TW MY KN CZ BUD Buffer of Uniform Data SC AR XX MB SEEDLink (5) Earthworm Waveserver (21) UU CH NL UW GE MN NI PBO UO PR PA WY NM IW ER ER RE CC PE ET IE LI US LB Real Time Data at IRIS
LISS SeedLink DHI BUD Buffer of Uniform Data Real Time Data Flow from DMC Antelope Earthworm SeisGram2K VASE SOD EMM