1 / 31

SXDS database and Japanese Virtual Observatory

SXDS database and Japanese Virtual Observatory. Yuji Shirasaki and JVO collaborations National Astronomical Observatory of Japan. What is Virtual Observatory ?.

skyler-lane
Download Presentation

SXDS database and Japanese Virtual Observatory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SXDS database and Japanese Virtual Observatory Yuji Shirasaki and JVO collaborations National Astronomical Observatory of Japan

  2. What is Virtual Observatory ? • Virtual Observatory (VO) is an collection of the astronomical DBs which are accessible with a standard protocol over the internet.

  3. HST XMM ALMA Astro-F Astro-E2 SDSS Subaru 2MASS • Avalanche of the observational data • Nobeyama ~ 1TB/year • Subaru ~ 20TB/year • ALMA ~ 1PB/year • XMM, Chandra, Astro-E2, Astro-F, ... • The data growth rate (T2 < 1 year) is higher than the improvement of computational power (T2 ~ 18 months) and network bandwidth (T2 ~ 20 months) the newest informational technology, Data Grid and Computational Grid. • A multiwavelength study is crucial for understanding the nature of astronomical objects, however... • Calibration and analysis methods are highly non-uniform across archives  manual data reduction is required.It is a physical and mental barrier to the multiwavelength study. Why do we need a Virtual Observatory ?

  4. Subaru ALMA Astro-E2 Astro-F XMM HST SDSS 2MASS SExtractor HyperZ IRAF ... What is enabled in the VO era ? Digital Universe is on your desktop ! • Easy and seamless access to a large number of databases.  You don’t need to access to tens or hundreds of web sites. • Automated data reduction.  You are free from calibration issues. • Federation of databases and analysis tools.  Efficient data analysis environment. • Discovery of rare and exotic objects based on multiwavelength observations of billions of objects instead of hundreds. • Discovery of transient phenomena exploring the time-domain. • Discovery of ... VO Request result federation of data and analysis services by VO standards

  5. Registry VOTable FITS Open Sky Server VO Query Language Source Extractor IRAF VLA Subaru Hyper-Z XMM Key components realizing the VO VO Query Language: Multi-Purposes Standard Query Language for VO DB DB Registry: Data & Analysis Service Discovery Service. DB VOTable: Tabular Data Transfer Format (XML) OpenSkyServer: VO Compliant Data Service.

  6. Resource metadata describes what data and computational facilities are available where, and once identified, how to use them. Resource Meta Data in Registry

  7. Astronomical Data Query Language (SQL) • Simple Image Access Protocol (URL based query) • Simple Spectrum Access Protocol (URL based query) Select a.* from Tab a where Region('Circle Cartesian 1.2 2.4 3.6 0.2') http://myimages.org/cgi-bin/VOimq?POS=180.,-30.&SIZE=0.0125... http://myspectrum.org/findSpectrum?POS=180.,-30.&SIZE=0.0125... VO Query Language defines a syntax for searching astronomical data. VO Query Language Unified Query Languagebased on JVO Query Language

  8. <?xml version="1.0"?><VOTABLE version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://vizier.u-strasbg.fr/xml/VOTable-1.1.xsd"><DEFINITIONS><COOSYS ID="J2000" equinox="2000." epoch="2000." system="eq_FK5"/></DEFINITIONS><RESOURCE name="myFavouriteGalaxies"><TABLE name="results"><DESCRIPTION>Velocities and Distance estimations</DESCRIPTION><PARAM name="Telescope" datatype="float" ucd="TEL_SIZE" unit="m“ value="3.6"/> <FIELD name="RA" ID="col1" ucd="POS_EQ_RA_MAIN" ref="J2000" datatype="float“ width="6" precision="2" unit="deg"/> <FIELD name="Dec" ID="col2" "POS_EQ_DEC_MAIN" ref="J2000" datatype="float“ width="6" precision="2" unit="deg"/> <FIELD name="Name" ID="col3" ucd="ID_MAIN" datatype="char" arraysize="8*"/> <FIELD name="RVel" ID="col4" ucd="VELOC_HC" datatype="int“ width="5" unit="km/s"/> <FIELD name="e_RVel" ID="col5" ucd="ERROR" datatype="int“ width="3" unit="km/s"/> <FIELD name="R" ID="col6" ucd="PHYS_DISTANCE_TRUE" datatype="float“ width="4" precision="1" unit="Mpc"> <DESCRIPTION>Distance of Galaxy, assuming H=75km/s/Mpc</DESCRIPTION> </FIELD> <DATA><TABLEDATA> <TR><TD>010.68</TD><TD>+41.27</TD><TD>N 224</TD><TD>-297</TD><TD>5</TD><TD>0.7</TD></TR> <TR><TD>287.43</TD><TD>-63.85</TD><TD>N 6744</TD><TD>839</TD><TD>6</TD><TD>10.4</TD></TR> <TR><TD>023.48</TD><TD>+30.66</TD><TD>N 598</TD><TD>-182</TD><TD>3</TD><TD>0.7</TD></TR> </TABLEDATA> </DATA></TABLE> </RESOURCE></VOTABLE> VOTable is an XML format for exchanging tabular data VOTable

  9. International Virtual Observatory Alliance (IVOA) http://www.ivoa.net/ 14 projects, ~$25 million in R&D an alliance of existing and future national and international projects to define standards on access to any kind of astronomical resources (database, analysis tools and so on...)

  10. Working Group in the IVOA • Registry : standardization of meta data • VOTable : define XML format for the exchange of tabular data • VO Query Language : standard query language for astronomical data base • Data Access Layer : define and formulate VO standards for remote data access. • UCD : defining and standardizing meta data (Unified Content Descriptors) • Data Modeling : the IVOA data modeling standardization • Grid & Web Service : Use of Grid technologies and Web Services in the VO context

  11. IVOA Interest Groups • Application:intended to support developers and users of Virtual Observatory applications • VO Theory: formed with the goal of ensuring that theoretical data and services are taken into account in the IVOA standards process Mailing List discussions: http://www.ivoa.net/forum/

  12. Interoperability Meeting aimed at making significant progress in generating new standards powering the development of the world wide Virtual Observatory initiatives. • 2003-05 Cambridge, UK • 2003-10 Strasbourg, France • 2004-05 Boston, USA • 2004-09 Pune, India • 2005-05? Japan

  13. Japanese Virtual Observatory • Takes the initiative of DB standardization in the Japanese astronomical community. • Nobeyama, Subaru and ALMA DB • Databases managed by ISAS and other institutes. • Contributes to standardization of VO query language. • Provides a VO portal where one can seamlessly access to the VO compliant data and analysis services.

  14. JVO Collaborators Project Scientists NAOJ • Mizumoto • Ohishi • Shirasaki • Tanaka • Honda ICRC • Yasuda Ochanomizu U. • Masunaga System Engineers Fujitsu Ltd. • Monzen • Kawarai • Ishihara • Yanaka • Yamaguchi • Ishida • Saito • Abe • Tsutsumi SEC Ltd. • Morita • Nakamoto • Kobayashi • Yoshida

  15. JVO Prototype • Experiment for DB federation • Functionality test of JVO Query Language • Globus Toolkits 3 • Remote processing and file transfer. • Web based User Interface • Tomcat and Struts • Distributed Databases • DB1  SXDS Suprime-Cam (Catalog & Image) • DB2  SXDS XMM (Catalog & Image) • DB3  SDF • DB4  SDSS Spectrum • DB5  2MASS

  16. Registry (XMLDB) Controller User Interface Grid Service • Parser • Scheduler • Executer • JVOQL Editor • VOTable Viewer • DB Search • Plotter • etc JVO Server Server 1 Grid Service Architecture of JVO Proto 2 Resource Metadata Meta DB • Data Search • JVOQL • Analysis • Parameters Catalog DB FITS JVOQL (VOTable) XML XPath VOTable FITS Server 2 • User input • Create an observation procedure • Resolve data service location using registry • Execute data search and/or data analysis • Save results in the user DB Grid Service JVOQL (VOTable) Meta DB VOTable FITS Catalog DB FITS Server 3 Parameter List User DB Grid Service XML etc. JVO Portal Server

  17. Data Server 1 (SXDS Subaru) Grid Service Meta DB Registry (XMLDB) Catalog DB FITS JVO Portal Server Data Server 2 (SXDS XMM) Grid Service Meta DB Catalog DB FITS Cross match of the optical and X-ray catalogs of SXDS and image retrievals. Demo 1: Cross match & Image request Registry Server Grid Service resolve data service URL cross match result (5) (2) (4) xmatch request (7) image (8) result image (6) (1) (3) Search Request (JVOQL) catalog search

  18. select opt.POS_EQ_RA_MAIN as ra, opt.POS_EQ_DEC_MAIN as dec, opt.N18APMAGB as mag_B, opt.N18APMAGR as mag_R, opt.N18APMAGi as mag_i, opt.N18APMAGz as mag_z, x.POS_EQ_RA_MAIN as ra_x, x.POS_EQ_DEC_MAIN as dec_x, x.flux0, x.flux1, x.flux2, x.flux3, x.flux4, img_opt.BOX(POINT(ra, dec), 20 arcsec, 20 arcsec) as image_opt, img_x.BOX(POINT(ra, dec), 20 arcsec, 20 arcsec) as image_x from naoj.sxds.sxdsR1 opt, naoj.xmm.xmm_epic_sxds x, naoj.sxds.sxds_image img_opt, naoj.xmm.xmm_image img_x where XMATCH(opt, x) < 5 arcsec and opt.N18APMAGR < 24 and BOX(POINT(34.5, -5.0), 0.1, 0.1) Sample Query optical catalog X-ray catalog optical image X-ray image Precision for the cross identification of the optical and X-ray objects. region selection

  19. Registry (XMLDB) JVO Portal Server Data Server 1 (SXDS Subaru) Grid Service Meta DB Catalog DB FITS Data request to the SXDS optical catalog, GL candidate selection, String search by pattern recognition. Demo 2: Cosmic String Search Registry Server Analysis Server Grid Service Grid Service resolve data service URL in each request String search (7) (8) result (5) GL cand. selection result (3) (1)(4)(6) (2) Data request GL candidate selection String search catalog search

  20. Prediction by Unified theory • super heavy cosmic strings with linear mass density of 1022 g/cm in the early universe. • The lens effect by a long cosmic string • undistorted lensed image • co-aligned in a direction of string network • distributed in a very large scale. • Because of its large scale nature, wide fied deep survey is crucial for its discovery.  Data mining from Subaru Suprime-Cam image data Cosmic String http://www.damtp.cam.ac.uk/user/gr/public/cs_evol.html

  21. User Authentification

  22. JVO QL Editor Create SQL Search

  23. Search Status

  24. Search Result

  25. Cosmic String Search Demo using SXDS Data base • Catalog Search • GL Candidate Search • Select pair objects of similar color. • Cosmic String Search • Pattern recognition

  26. Cosmic String Search Result

  27. Summary Road map of the JVO project 2004 Prototype 3 • development of components for operational system • Subaru Suprime-Cam reduced data DB • ISAS DARTS ? • Test connection to the international VO 2005 start to develop operational system • late 2005 ?? trial use 2006-2007 trial use & upgrade 2007 prepare for partial operation of ALMA

  28. Nobeyama ALMA Subaru

  29. YOHKO Astro-F Astro-E HALCA

  30. Select objects located in a circle region centered at ra=270 and dec=-1.5 with 0.2 deg radius from optical catalog. • Identify X-ray counter part for each selected object with 5 arcsec precision. • Get FITS images of 10” radius size from optical and X-ray image data service. JVO Query Language select optCat.ra, optCat.dec, xCat.ra, xCat.dec, optImage.Image, xImage.Image from optCat, xCat, optImage, xImage where ((‘ICRS’, 270.0 deg, -1.5 deg), 0.2 deg) ~ (optCat.ra, optCat.dec) andDistance((optCat.ra, optCat.dec), (xCat.ra, xCat.dec)) < 5 arcsec and optImage.regionSky= ((optCat.ra, optCat.dec), 10 arcsec) and xImage.regionSky= ((optCat.ra, optCat.dec), 10 arcsec) and optImage.FORMAT= “FITS” and xImage.FORMAT= “FITS”

  31. DB Search

More Related