1 / 16

So the data are in the db(s)…

So the data are in the db(s)…. Now what?. NSF AS. So the data are in the db(s)…. Now what?. G. dbBadger (data pig?). Issues. For visualization, playing, and analysis, data need to be synchronized SQL doesn’t deal well with many situations Crossing databases

akiko
Download Presentation

So the data are in the db(s)…

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. So the data are in the db(s)… Now what? NSF AS

  2. So the data are in the db(s)… Now what? G dbBadger (data pig?)

  3. Issues • For visualization, playing, and analysis, data need to be synchronized • SQL doesn’t deal well with many situations • Crossing databases • Asynchronous data are troublesome • Data with different time formats are buggers • Synchronization, other than aggregation a pain • Need a tool that can request data, synchronize, and interface with other programs

  4. dbBadger • Define the universe • Connects simultaneously to multiple databases • Select the variables • Allows any combination of time-based variables • Assumes all variables have a time component • No requirement of relationship among variables • Aggregate, using any of 3 • Union • Interpolate • Aggregate • Export results • Additional signal processing • Other software

  5. dbBadger – Other Features • Graphics interface or command line • Runs w/in Matlab or stand-alone • Freely distributable • Savable state for easy reruns

  6. 1. Locations 2. Tables 3. Fields 4. Selections 5. Date range 6. Get data 7. Additional functions {

  7. 1. Locations 2. Tables 3. Fields 4. Selections 5. Date range 6. Get data 7. Additional functions {

  8. dbBadger Output

  9. 4. Selections 7a. Aggregate 7b. Save

  10. Interaction with other software dbBadger dbBadger dbBadger db Oracle in Madison Oracle in Taiwan Access on desktop • Pass: • Config file • New date • Output location model

  11. Issues • Speed • Easily beats SQL • High-resolution data slow • Standardization • Measurement naming standards • Routing w/in database • Interoperability • Other software • Server mode • Emphasis • Currently, location • Future, measurement variable

  12. 1 month HIRES WTEMP Trout Lake Retrieval time: 15sec for ~200,000 records 1 month HIRES WTEMP YYL Retrieval time: 15 minutes Data Retrieval Speed

  13. Measurement Defs. Sever • Time • Integrated • MM:DD:YY:HH:SS • Separated • NTL • Year • DayOfYear • Military • Temperature • Dissolved oxygen • Winkler • Sensor • YSI Model 58 • Greenspan • Optical • etc. Database generator Database dbBadger

  14. Database • What variables are measured? • What are their locations (3D coordinates)?

More Related