1 / 16

Sky Surveys and the Virtual Observatory

Sky Surveys and the Virtual Observatory. Alex Szalay The Johns Hopkins University. Living in an Exponential World. Astronomers have a few hundred TB now 1 pixel (byte) / sq arc second ~ 4TB Multi-spectral, temporal, … → 1PB

glenna
Download Presentation

Sky Surveys and the Virtual Observatory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sky Surveys and the Virtual Observatory Alex Szalay The Johns Hopkins University

  2. Living in an Exponential World • Astronomers have a few hundred TB now • 1 pixel (byte) / sq arc second ~ 4TB • Multi-spectral, temporal, … → 1PB • They mine it looking fornew (kinds of) objects or more of interesting ones (quasars), density variations in 400-D space correlations in 400-D space • Data doubles every year • Caused by the emergenceof generations ofinexpensive sensors + computing

  3. Why Is Astronomy Special? • Especially attractive for the wide public • Community is not very large • It has no commercial value • No privacy concerns, freely share results with others • Great for experimenting with algorithms • It is real and well documented • High-dimensional (with confidence intervals) • Spatial, temporal • Diverse and distributed • Many different instruments from many different places and many different times • The questions are interesting • There is a lot of it (soon petabytes)

  4. Sloan Digital Sky Survey Goal Create the most detailed map of the Northern sky “The Cosmic Genome Project” Two surveys in one Photometric survey in 5 bands Spectroscopic redshift survey Automated data reduction 150 man-years of development High data volume 40 TB of raw data 5 TB processed catalogs Data is public 2.5 Terapixels of images The University of Chicago Princeton University The Johns Hopkins University The University of Washington New Mexico State University Fermi National Accelerator Laboratory US Naval Observatory The Japanese Participation Group The Institute for Advanced Study Max Planck Inst, Heidelberg Sloan Foundation, NSF, DOE, NASA

  5. The Photometric Survey Continuous data rate of 8 Mbytes/sec Northern Galactic Cap drift scan of 10,000 square degrees 5 broad-band filters exposure time: 55 sec pixel size: 0.4 arcsec astrometry: 60 mas calibration: 2% at r'=19.8 done only in best seeing (20 nights/year) Southern Galactic Cap multiple scans (> 30 times) of the same stripe u‘ g' r‘ i ' z’ 22.3 23.3 23.1 22.3 20.8

  6. Survey Strategy Overlapping 2.5 degree wide stripes Avoiding the Galactic Plane (dust) Multiple exposures on Southern stripes

  7. The Spectroscopic Survey • SDSS Redshift Survey • 1 million galaxies • 900,000 r’ limited • 100,000 red galaxies • volume limited to z=0.45 • 100,000 quasars • 100,000 stars • Two high throughput spectrographs • spectral range 3900-9200 Å • 640 spectra simultaneously • R=2000 resolution, 1.3 Å • Features • Automated reduction of spectra • Very high sampling density and completeness • Objects in other catalogs also targeted

  8. Data Processing Pipelines

  9. Analyzing the SkyServer • Prototype in data publishing • 350 million web hits in 6 years • 930,000 distinct usersvs 10,000 astronomers • Delivered 50,000 hoursof lectures to high schoolstudents • Delivered 100B rows of data • Everything is a power law • GalaxyZoo • 27 million visual galaxy classifications by the public • Enormous publicity (CNN, Times, W.Post, BBC) • 100,000 people participating

  10. Skyserver Sessions Singh et al (2007)

  11. Trends CMB Surveys (pixels) • 1990 COBE 1000 • 2000 Boomerang 10,000 • 2002 CBI 50,000 • 2003 WMAP 1 Million • 2008 Planck 10 Million Angular Galaxy Surveys (obj) • 1970 Lick 1M • 1990 APM 2M • 2005 SDSS 200M • 2008 VISTA 1000M • 2012 LSST 3000M Time Domain • QUEST • SDSS Extension survey • Dark Energy Camera • PanStarrs • SNAP… • LSST… Galaxy Redshift Surveys (obj) • 1986 CfA 3500 • 1996 LCRS 23000 • 2003 2dF 250000 • 2005 SDSS 750000 Petabytes/year by the end of the decade…

  12. National Virtual Observatory • NSF ITR project, “Building the Framework for the National Virtual Observatory” is a collaboration of 17 funded and 3 unfunded organizations • Astronomy data centers • National observatories • Supercomputer centers • University departments • Computer science/information technology specialists • Similar projects now in 15 countries world-wide => International Virtual Observatory Alliance

  13. Continuing Growth How long does the data growth continue? • High end always linear • Exponential comes from technology + economics  rapidly changing generations • like CCD’s replacing plates, and become ever cheaper • How many new generations of instruments are left? • Are there new growth areas emerging? • Software (collaboration) is becoming an instrument • hierarchical data replication • Value added data/ mashups • data cloning

  14. Technology+Sociology+Economics • Neither of them is enough • We have technology changing very rapidly • Sensors, Moore's Law • Trend driven by changing generations of technologies • Sociology is changing in unpredictable ways • In general, people will use a new technology if it is • Offers something entirely new • Or substantially cheaper • Or substantially simpler • Funding is essentially level

  15. Surveys from Arecibo • Facility very suitable for observing known sourcesor with known redshift in the ’local’ universe • Perfect match to SDSS galaxies (<z>~0.1) • Matching sky coverage of SDSS, GALEX and FIRST • Enormous benefit from using new detector technologies (focal plane phased array) • Possible large scale survey projects: • Virgo Deep • SDSS+GALEX HI survey • Integrated HI in distant clusters (vs z) • Local universe (<10,000 km/s)

  16. Summary • Science is aggregating into ever larger projects • Collection of data is separating from science analysis • Much of recent evolution is through new sensors • VO is inevitable, a new way of doing science • Present on every physical scale today, not just astronomy (Earth/Oceans, Biology, MS, HEP) • Might be the only way to do 'small science' in 2020

More Related