1 / 74

Hello!

Hello!. Virtual Observatories. Ajit Kembhavi IUCAA Pune, India. Data Storage and Retrieval. The Astronomer Vermeer 1632-1675. The Library of Alexandria 3 rd Century BC. The Data Avalanche. Immense amounts of data are being produced by large telescopes using large area detectors.

vina
Download Presentation

Hello!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hello!

  2. Virtual Observatories Ajit Kembhavi IUCAA Pune, India

  3. Data Storage and Retrieval The Astronomer Vermeer 1632-1675 The Library of Alexandria 3rd Century BC

  4. The Data Avalanche Immense amounts of data are being produced by large telescopes using large area detectors. Terabytes of data are now available, and Petabytes will soon be available from frequent all sky imaging. Vast databases are also being produced through simulations.

  5. Wavelength Coverage The data spans the electromagnetic spectrum from the radio to the gamma-ray region. Obtaining, analysing and interpreting the data in different wavebands involves highly specialised instruments and techniques. The astronomer needs new tools for using this wealth of data in multiwavelength studies.

  6. Stars in the Milky Way

  7. The Hertzsprung-Russell Diagram

  8. The Alliance Members of the IVOA

  9. Interactions

  10. Virtual Observatories • Provide tools for data analysis, visualization and mining. • Develop interoperability concepts to make different databases seamless. • Manage vast data resources and provide these on-line to astronomers and other users. Empower astronomers by providing sophisticated query and computational tools, and computing grids for producing new science.

  11. VO Schema

  12. IVOA Technology Initiatives The IVOA has identified six major technical initiatives to fulfill the scientific goal of the VO concept. IVOA-LISTS

  13. REGISTRIES: These collect metadata about data resources and information services into a queryable database. The registry is distributed. A variety of industry standards are being investigated. DATA MODELS: This initiative aims to define the common elements of astronomical data structures and to provide a framework to describe their relationships. UNIFORM CONTENT DESCRIPTORS: These will provide the common language for for metadata definitions for the VO.

  14. DATA ACCESS LAYER:This provides a standardized access mechanisms to distributed data objects. Initial prototypes are a Cone Search Protocol and a simple Image Access Protocol. VO QUERY LANGUAGE:This will provide a standard query language which will go beyond the limitations of SQL. VOTable: This is an XML mark-up standard for astronomical tables.

  15. Science Initiatives • Many IVOA projects have active Science Working Groups consisting of astronomers from a broad cross-section of the community representing all wavelengths. • The focus here is to develop a clear perception of the scientific requirements of a VO. • Projects within the working groups will develop new capabilities for VO based analysis. • This will enable the community to create new research programs and to publish their data and research in a more pervasive and scientifically useful manner.

  16. Virtual Observatory -India A collaboration between IUCAA and PSPL, with a grant from the Ministry of Communications and Information Technology

  17. IUCAA

  18. Persistent Systems Pvt. Ltd., Pune

  19. Virtual Observatory - India

  20. Data Archives and Mirrors at VO-I SDSS 2Mass 2DFGRS 2QZ FIRST NVSS Chandra Vizier, Aladin, ADS

  21. Fast Computing • Four Alpha Server ES-45 machines • Each with 4 processors Alpha 21264C • RAM 3 x 8 GB + 12 GB • Fast, Low latency interconnect • Memory channel Architecture (MCA) • 1 TB SCSCI storage • Trucluster clustering environment • (Tru64 Unix, DecMPI, openMP)

  22. VO-India Software Projects VOPlot Visualizer for catalogue data VOTable C++ Parser VOTable Streaming writer Data Converters Fits Browser User interfaces and query tools Applications beyond astronomy All tools have web-based and stand alone versions

  23. The VOPLOT Collaboration Visualization and simple statistics of catalogue data. Integration with sky atlases.

  24. The VOPlot Collaboration A VO-I + CDS collaboration First conceived as a web-based tool for Vizier Then integrated with Aladin VOPlot is now also a stand alone system It has been integrated with many data bases Sonali Kale, K.D. Balaji et. Al. VOPlot

  25. SDSS J125637-022452 High proper motion L-subdwarf Optical spectra of mixed late M and mid L type Only the third L subdwarf known

  26. Positions 1986-2000 Proper motion 0.617 arcsec / yr

  27. AVO Prototype Demo Astrogrid: Astronomy Catalogue Extractor AVO: Aladin+SED VO-India:VOPlot

  28. FITS Manager View, create and add to FITS files Convert to other formats Pallavi Kulkarni Fits-manager

  29. VOTable Java Streaming Writer Acts on a data array in memory to convert it to the VOTable form, which is streamed row by row to an output file. Very large VOTables can be written without excessive memory. Pallavi Kulkarni VOTable-Java

  30. Thank You

  31. VOTable • This is a new data exchange standard produced through efforts led by Francois Ochsenbien of CDS, Strasbourg and Roy Williams of Caltech. • VOTable is in XML format. Physical quantities come with sophisticated semantic information.

  32. VOTable • The format enables computers to easily parse the information and communicate it to other computers. • Federation and joining of information become possible and Grid computing is easier. • VOTable parsers have been developed in Perl, Java and C++. • Enhancements and extensions are being considered. Streaming Parser Non-streaming Parser

  33. VOTable Data The data part in a VOTable may be represented using one of three different formats: • FITS : VOTable can be used either to encapsulate FITS files, or to re-encode the metadata. • BINARY : Supported for efficiency and ease of programming, no FITS library is required, and the streaming paradigm is supported. • TABLEDATA : Pure XML format for small tables.

  34. C++ VOTable Parser Motivation: • Provide a library for API based access to VOTable files. • APIs can be directly used to develop VOTable applications without having to do raw VOTable processing. • Streamingand Non-streaming versions are available. Sonali Kale, Sudip Khanna

  35. C++ VOTable Parser Salient Features: • Implemented as a wrapper over XALAN-C++. • XALAN-C++ is a robust implementation of the W3C recommendations for XSL Transformations (XSLT) and the XML Path language (XPath). • XPath queries can be used to access the VOTable data.

  36. Project Design VTable Metadata Link Collection Link Field Collection Field LinkCollection Link TableData Values minimum RowCollection maximum Row OptionCollection ColumnCollection Options

  37. IUCAA HPC Facility Hercules HPC Team : Sarah Ponthratnam Sunu Engineer Rajesh Nayak Anand Sengupta Co-proposed by : Ajit Kembhavi T. Padmnabhan Tarun Souradeep • Four Alpha Server ES-45 machines • Each with 4 processors Alpha (21264C) • 1.25 GHz clock speed • Cache on chip: 64 Kb –I, 64 Kb-D • Cache :16 Mb ECC DDR • RAM 3 x 8 Gb + 12 Gb • Fast, Low latency interconnect • Memory channel Architecture (MCA) • High volume Storage • 1 Tera-byte SCSCI • Trucluster clustering environment (Tru64 Unix, DecMPI, openMP) > 30 G flops Preliminary HPL benchmark ES-45 Specfp2000: 1327 Linpack 1000x1000: 6847

  38. Virtual Observatory - India Persistent Systems IUCAA

More Related