1 / 28

HEASARC Status and Plans

HEASARC Status and Plans. Nick White. The HEASARC Charter. Established December 1990. Maintain and disseminate data from previous and concurrent high-energy astrophysics missions Provide software and data analysis support for these data sets

donoma
Download Presentation

HEASARC Status and Plans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HEASARC Status and Plans Nick White Nick White

  2. The HEASARC Charter Established December 1990 • Maintain and disseminate data from previous and concurrent high-energy astrophysics missions • Provide software and data analysis support for these data sets • Maintain and provide the necessary scientific and technical expertise for the processing and interpretation of the data holding • Develop and maintain multi-mission analysis and support tools • Provide catalogs of observations and ancillary information for the data holdings • Coordinate data, software and media standards with other astrophysics sites • Support education and outreach activities in high energy astrophysics (not part of original charter) Nick White

  3. The HEASARC and High Energy Astronomy Missions GLAST Swift 2 HETE2 Integral Chandra data@CXC HEASARC provides the multi-mission infrastructure that is used by parallel GOFs and science support centers in the LHEA and elsewhere Nick White

  4. HEASARC Science Personnel • Nicholas White: HEASARC Head • Steven Murray: HEASARC Deputy Head (at CXC) • Lorella Angelini: Data Restoration, Image and timing analysis, BeppoSAX, Swift (inc. AKBAR), Astro-E2 • Keith Arnaud: Data selection software, spectral analysis • Michael Corcoran: Calibration, GLAST, HETE-2, ROSAT • Stephen Drake: Catalogs, website & archive quality assurance, EUVE, RXTE, CGRO • James Lochner: Education and public outreach • Thomas McGlynn: Archives, Virtual Observatory, SkyView • Bill Pence: FITS, Software, HEASOFT, Hera Nick White

  5. Recent HEASARC Highlights • Hardware: • Acquisition of 8.4 TB (12 TB raw) TB network appliance (March 2003) • Upgrade to Gigabit network (Oct 2003) • Science Archive: • XMM downloads now 2nd in volume to RXTE • HETE-2 archive reaches 150 GB and 51 bursts • Specialized Swift Browse interface • Software: • HEAsoft 5.3 release (November 2003): Improved fv integration with DS9 • Browse 7.0 release (January 2004): new upload & download features • XSPEC 12 testing by Integral SPI team (ongoing) • Catalogs: • GRBCAT master catalog of gamma-ray bursts (Feb 2004) • Education and Outreach • SkyView SciAm Top 50 award (June 2003: 2nd year in a row) • Teacher’s Hera beta implementation (July 2003) • Services: • First VO Tool: Data Inventory Service (July 2003) Nick White

  6. The Physical Archive Active Missions RXTE (1995- ) Chandra (1999- ) [data at CXC] HETE-2 (2000- ) Integral (2001-) XMM-Newton (1999- ) Past Missions Ariel 5 EXOSAT ASCA Ginga BBXRT BeppoSAX CGRO HEAO 1 Copernicus HEAO 3 COS B OSO 8 DXS ROSAT Einstein SAS 2 EUVE SAS 3 Vela 5B Upcoming Missions Swift (2004 Launch) Astro-E2 (2005 Launch) GLAST (2007 Launch) • Data from 24 missions currently in the archive • 355 astronomical catalogs & mission tables • The archive volume was 3250 Gigabytes as of the end of 2003 Nick White

  7. Usage & Data Statistics Gigabytes transferred per year 1998 1999 2000 2001 2002 2003 Datasets by ftp & http 844 1406 1880 2270 2917 3943 Science Web Pages & Images 198 234 371 399 580 872 Main E&PO Web Pages & Images 203 358 678 1010 3108 7581 Nick White

  8. Data Transfers by Mission over Time Nick White

  9. Current Web Service Statistics • Archive volume: 3.3 TB (mostly compressed) • Current ingest rate: 400 GB/yr • End of 2004 anticipated rate: 1.4 TB/year, with increased or new flows from Swift, Integral, and XMM • Missions in archive: Ariel-5, ASCA, BBXRT, BeppoSAX, Chandra, Compton, Copernicus, COS-B, DXS, Einstein, EUVE, EXOSAT, Ginga, HEAO-1, HEAO-3, HETE-2, OSO-8, ROSAT, RXTE, SAS2, SAS3, Vela 5B, XMM-Newton • Downloads (annual): 3.9 TB of data (by FTP and HTTP), 0.9 TB of science web pages (by HTTP) and 7.6 TB of E&PO web pages • Browse: 355 local tables (+ ~3000 VizieR); 2,150,000 queries • SkyView: >60 surveys; >1,000,000 images generated • 26,000 bibliography entries linking ADS to HEASARC datasets (ROSAT, ASCA, XMM-Newton, and some RXTE datasets) Nick White

  10. 12 TB Network Appliance Archive 750 GB NetApp Static web site and shared disks DLT Jukebox Backup archive on 150 GB DLT tapes (copy to NSSDC) Internal Machines DBMS1 on line DBMS2 backup HEASARC Hardware Configuration Web servers HEASARC1 2 GHz Dual Processor GSFC Portal HEASARC (load balancer) HEASARC2 2 GHz Dual Processor More slots available 100 Mb Database (Sybase) 2 GHZ dual Processors SkyView (x2) 1 GHz Hera (x5) 1 GHz Gigabit Online archive backup 6 TB Assorted Storage Safehouse 400 GB User Cache 150 GB Surveys (x2) 240 GB External Survey Cache Linux, Solaris, Mac, Windows, … Nick White

  11. Web site maintenance • Main HEASARC web site has 15,000 static pages and hundreds of dynamic pages (excluding the FTP area and data archive) comprising 7 GB in total • Ensuring valid and up-to-date links pages requires constant attention • In the past, invalid links left unattended grew to ~500. • Concentrated effort has now eliminated all but a few broken or out of date links (Newly broken links are fixed weekly) • HEASARC is now 99% Section 508 compliant • Assuring appropriate access to HEASARC Web pages involved changes to most Web pages • Compliance changes were often beneficial for all users. • Maintenance cost of 508 compliance is <=0.05 FTE effort Nick White

  12. Browse Catalog and Archive Interface • Capable of querying all local and VizieR catalogs • Local and remote archives (e.g., CXC) • HTML, text, VOTable, FITS and Excel ouputs • New capabilities: • Expressions allowed in search criteria ( fluxX/fluxY > 2) • Better integration with SkyView • Upload files of targets (in Browse 7.0) • Download queries for later re-use (7.0) • Dataset ID info added when available (7.0) • Links between tables Nick White

  13. Current Active Missions • Chandra • Browse service fully linked to CXC archive • RXTE • Dominates current archive downloads in terms of GB • Simple XTEIndex table developed and deployed (more simplification needs to be done) • HETE2 • Some HEASARC-written FITS products for GRBs • Difficult to use most data delivered to archive • Similarities to data recovery efforts of early HEASARC • XMM-Newton • In 2003, second biggest HEASARC mission in archive volume & downloads • Archive contains both US & non-US Guest Observer data • Integral • Use modified version of HEASARC archive software at ISDC • HEASARC Browse can link to ISDC Browse tables • Archive at HEASARC will open in July 2004 Nick White

  14. Preparations for Swift • New HEASOFT package • MOU delineating HEASARC responsibilities • End to end testing including archive ingest. • Customized mission interfaces: • Individualized interface to Browse • HTML wrappers for standard Browse scripts • Links between Browse tables • AKBAR: new service to import external information from observers • HEASARC responsible for some high-level data products – e.g., overall burst evolution • Mission requires rapid dissemination of data Nick White

  15. Swift Browse Interface Nick White

  16. Preparations for GLAST • PDMP review • Consultations on archive and catalog design • Review proposed FITS formats Nick White

  17. Standards • FITS • Subsumed GSFC FITS Office Web site • Develop and coordinate domain conventions for high energy data (OFWG)) • Organize periodic HEADCC meetings • FITS Libraries Support • CFITSIO is essentially reference implementation of FITS • HEASARC also supports widely used FITS libraries in Java and IDL. • New C++ object implementation based on CFITSIO • Standard Dataset IDs • VO Standards • Browse support for VO Cone search • SkyView support for VO Simple Image Access Protocol • VOTable (and FITS) outputs available through Browse • VO Service Metadata descriptions for HEASARC resources Publication of metadata using Open Archives Initiative (OAI) protocol Nick White

  18. Software Tools and Services • HEASARC software is frequently downloaded • fv (FITS browser and editor): 500 downloads/month • CFITSIO library: 500/month • HEASOFT: 200-300/month • HEASOFT • New packages for Swift and Astro E2 missions under active development • Use HEADAS in order to provide a simple platform-independent procedure for new software builds and updates • XSPEC New major release (12.0) in alpha test • Complete re-write of Xspec internals in object oriented fashion • Can support coded aperture mask data sets Nick White

  19. Hera • Provides a full, on-line analysis environment to users. • Directly connected to archive, no downloads needed • Up to date versions of HEASOFT tools automatically available • Easily upload user data and download results. • Uploaded data screened to ensure service is not used inappropriately. • Uses standard HEASARC FV FITS browser tool as user interface. • Why? • Large volume of data (e.g., Integral, RXTE) downloads and/or software installation may be infeasible for some users • With Hera, users can start using HEAsoft and other software without the need for local installation • Status • Initial system operational and being used actively. • Proposal for continued funding has been submitted • Both teacher and student interfaces in beta testing Nick White

  20. New and Updated Catalogs • ~24 new catalogs or major catalog updates in last year. • Major correlative catalogs: VERONCAT, GB6, etc • High energy source catalogs: XMMSSC, BMWHRICAT, INTREFCAT • Master catalogs updated regularly • GRBCAT: Compilation of all gamma-ray bursts • VizieR connection for transparent access to massive catalogs (2MASS, USNO B, GSC 2) Nick White

  21. Education and Outreach • Education Web Sites: • Imagine the Universe (Grades 6-12): 500K hits/month. • StarChild (Grades 1-8) [in collaboration with L. Whitlock]: 1.25 M hits/month. • New poster on origin of the elements • X-ray analysis school • Highly successful and oversubscribed • Host for APOD Web site • StarChild/Imagine/APOD CDs Nick White

  22. Other Projects • SkyView • Java redesign • New surveys (DSS2, Halpha, SFD) & SDSS soon. • Request for continued funding submitted • ClassX • Automated classification of entire RASS and WGACAT samples of X-ray sources using supervised learning algorithm Density of source requests in SkyView Nick White

  23. Virtual Observatory Involvement • Data Inventory Service • HEASARC metadata inventory • Standardized metadata describing all HEASARC tables,data file types, surveys and services. • Registered with VO to enable effective discovery of HEASARC resources. • Table metadata • HEASARC support for VO standards Nick White

  24. Data Inventory Service • Access to distributed services that use VO protocols. • Many sites contributing data • Dynamic service lookup using VO registries • Catalog, observations, Images http://heasarc.gsfc.nasa.gov/vo/ Nick White

  25. Near Term Missions and Deadlines • Launches • Swift: Fall 2004 • Astro-E2: Early 2005 • GLAST: Early 2007 • HEASARC Senior Review Proposal Due: March 17, 2004 Nick White

  26. Current and Upcoming Issues Science • May need greater emphasis on supporting gamma-ray astronomers as Integral and GLAST Archives come online and become an increasing fraction of the HEASARC’s archive • Must deal with ever increasing complexity of data and calibration • Coded aperture masks (Swift, HETE2, Integral) • Complex source dependent response (“ + GLAST) • Multi-satellite observations (Con-X, LISA) Nick White

  27. Current and Upcoming Issues Hardware • CPU and disk space needs achievable, but bandwidth is a potential limitation • Datasets may be growing faster than bandwidth • Demand for data will spike after GRB detections • Higher speed linkage to Internet possible but costly • … and doesn’t help users who have slower connections • Move towards more user processing at HEASARC • Hera, grid computing • On-line backup for archive is presently in same building as primary HEASARC archive. • Planned move of backup archive to another building when space and network is ready (Summer 2004) Nick White

  28. Current and Upcoming Issues Local Software and Services • Integration of tools and services • Strengthen SkyView/Browse, Browse/NVODIS, Hera/Browse, … cross-links. • Enable coupling archive/web services and HEASOFT tools. • Systematic attention to internal documentation of HEASARC systems Nick White

More Related