1 / 18

Fermi Large Area Telescope Tutorial on Science Analysis Data Access Tony Johnson

Fermi Gamma-ray Space Telescope. Fermi Large Area Telescope Tutorial on Science Analysis Data Access Tony Johnson Stanford Linear Accelerator Center tonyj@slac.stanford.edu. http://glast-ground.slac.stanford.edu/DataPortal. Data Portal Components. The Data Portal Comprises: Astro Server

dawn
Download Presentation

Fermi Large Area Telescope Tutorial on Science Analysis Data Access Tony Johnson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fermi Gamma-ray Space Telescope Fermi Large Area Telescope Tutorial on Science Analysis Data Access Tony Johnson Stanford Linear Accelerator Center tonyj@slac.stanford.edu http://glast-ground.slac.stanford.edu/DataPortal

  2. Data Portal Components • The Data Portal Comprises: • Astro Server • Common History • Fits Skimmer • Merit Skimmer • Data Catalog • Web Interface • Linemode Client • Download Manager • Web Interface • Command Line • WIRED Event Display • This talk will only cover the items in red. • All components available via the web interface at: • http://glast-ground.slac.stanford.edu/DataPortal

  3. “Astro Server”

  4. Astro Server • Allows data to be selected based on any combination of: • Energy Range • Position in sky (ra,dec,radius) • Position can be looked up in SIMBAD catalog • Event Class (transient, source, diffuse) • Time Range (MET) • Can produce FT1 (fits), FT2 (fits) and/or Merit (root)* • Supports multiple “Event Samples” • Currently “LEOScience” or “Level1” • Can support access to multiple versions of data once we start reprocessing, or if we want to have any kind of special event selections • Level1 data currently loaded as soon as it becomes available from pipeline • Web and command line interfaces available * Coming Soon

  5. Astro Server Web Interface Page pre-populated with your most recent selection You can give job meaningful name Leave fields blank for “no-limit” Give name of astronomical object, sirius, M31, MCG+02-60-010 Expert options allow experimental features to be easily added

  6. Astro Server Web Interface… README contains job parameters

  7. Common History • Data Portal History can be used: • Show status of all your skimmer/astro server jobs • To access output directory, pipeline details for all jobs • Recover parameters of any job Tutorial Demo Click to go to output directory Click to go to pipeline Click to recover parameters

  8. Common History… • Clicking on job name in history recovers parameters • Allows rerunning same or similar job

  9. “Fits Skimmer”

  10. Fits Skimmer • Supports merging any FT1/FT2* files in data catalog • Allows “fcopy rowfilter” style cuts to be applied to FT1 columns • Example: • ZenithAngle<105 && CTBClassLevel==3 • http://heasarc.nasa.gov/lheasoft/ftools/headas/rowfilter.html • Allows selection of data based on meta-data in data catalog • Examples • sIntent==“nomSciOps” • LEOScience=="1835Survey“ • nMootKey==2103 || nMootKey==2104 • Outputs FT1 and FT2* files * Coming Soon

  11. Fits Skimmer Web Interface Page pre-populated with your most recent selection You can give job meaningful name Row Filter Data Selection (this is the selection Richard has been using for making merged FT1/FT2 files)

  12. Fits Skimmer Web Interface…

  13. Fits Skimmer vs. Astro Server • Astro Server • Allows cuts on energy, time, position, event class • Suitable for selecting limited number of events • Can output FT1, FT2, Merit* • Can only be used with data loaded into database • Uses database for efficient access to selected events • Fits Skimmer • Allows cuts on any FT1 column • Suitable for merging many files with minimal cuts • Can output FT1, FT2* • Can be used with any dataset in data catalog • Fully reads all input files • In future smarter single web interface may automatically choose most appropriate backend tool. * Coming Soon

  14. “Data Catalog”

  15. Data Catalog • Allows access to many datasets including: Level1 data, ASP data, Reprocessed data, MC data sets Key locations allow attention to be drawn to important datasets

  16. Data Catalog… • Data Catalog has been designed to be extensible • Can science groups make better use of data catalog? • Use it to define standard datasets? • As alternative to sending e-mail with location on user disk • See LEOScience folder as example • Use meta-data to keep track of common data selections • E.g. goodData==true instead of complex set of sIntent cuts

  17. Status • Working on improving documentation • Adding FAQ’s at: • https://confluence.slac.stanford.edu/display/ds/Data+Access+FAQ • Please contribute your own questions (we will try to provide the answers) • Data Portal is now mostly complete • Most of the planned functionality is available • Documentation is improving • We need to work with Science Groups to make sure we are making optimal use of the available tools • We need input from Science Groups (=you) on what additional features are needed/desired • E-mail me, or use helpsoftlist@glast.stanford.edu

  18. Discussion Items • Astro server • What data should be loaded? • Currently loaded with all FT1 (including LEO) satisfying “Richard’s” sIntent cuts • Should we remove early data? • Is this is the “right” set of data? • Currently all “transient, source, diffuse” data classes • Original plan was for less data, may need to beef up server as we get more data • Weekly summary DST files? • What data, when/how to make them? • Can science groups make better use of data catalog? • Use it to define standard datasets? • Use meta-data to keep track of common data selections

More Related