Mapping the Universe: The SDSS Project

1. The Sloan Digital Sky Survey Alex Szalay Department of Physics and Astronomy The Johns Hopkins University and the SDSS Project

2. The Sloan Digital Sky Survey A project run by the Astrophysical Research Consortium (ARC) The University of Chicago Princeton University The Johns Hopkins University The University of Washington Fermi National Accelerator Laboratory US Naval Observatory The Japanese Participation Group The Institute for Advanced Study Max Planck Inst, Heidelberg SLOAN Foundation, NSF, DOE, NASA Goal: To create a detailed multicolor map of the Northern Sky over 5 years, with a budget of approximately $80M Data Size: 40 TB raw, 1 TB processed Alex Szalay, JHU

3. Scientific Motivation Create the ultimate map of the Universe: The Cosmic Genome Project! Study the distribution of galaxies: What is the origin of fluctuations?  What is the topology of the distribution? Measure the global properties of the Universe: How much dark matter is there? Local census of the galaxy population: How did galaxies form? Find the most distant objects in the Universe: What are the highest quasar redshifts? Alex Szalay, JHU

4. daCosta etal 1995 SDSS Collaboration 2002 deLapparent, Geller and Huchra 1986 Gregory and Thompson 1978 The Cosmic Genome Project The SDSS will create the ultimate mapof the Universe, with much more detailthan any other measurement before Alex Szalay, JHU

5. Area and Size of Redshift Surveys Alex Szalay, JHU

6. Clustering of Galaxies We will measure the spectrum of the density fluctuations to high precision even on very large scales The error in the amplitude of the fluctuation spectrum 1970 x100 1990 x2 1995 ±0.4 1998 ±0.2 1999 ±0.1 2002 ±0.02 Alex Szalay, JHU

7. Finding the Most Distant Objects Intermediate and high redshift QSOs Multicolor selection function. Luminosity functions and spatial clustering. High redshift QSO’s (z>5). Alex Szalay, JHU

8. Features of the SDSS Special 2.5m telescope, located at Apache Point, NM 3 degree field of view. Zero distortion focal plane. Two surveys in one: Photometric survey in 5 bands. Spectroscopic redshift survey. Huge CCD Mosaic 30 CCDs 2K x 2K (imaging) 22 CCDs 2K x 400 (astrometry) Two high resolution spectrographs 2 x 320 fibers, with 3 arcsec diameter. R=2000 resolution with 4096 pixels. Spectral coverage from 3900Å to 9200Å. Automated data reduction Over 100 man-years of development effort. (Fermilab + collaboration scientists) Very high data volume Expect over 40 TB of raw data. About 1 TB processed catalogs. Data made available to the public. Alex Szalay, JHU

9. Apache Point Observatory Located in New Mexico, near White Sands National Monument Alex Szalay, JHU

10. The Telescope Special 2.5m telescope 3 degree field of view Zero distortion focal plane Wind screen moved separately Alex Szalay, JHU

11. The Photometric Survey Northern Galactic Cap 5 broad-band filters ( u', g', r', i', z’ ) limiting magnitudes (22.3, 23.3, 23.1, 22.3, 20.8) drift scan of 10,000 square degrees 55 sec exposure time 40 TB raw imaging data -> pipeline -> 100,000,000 galaxies 50,000,000 stars calibration to 2% at r'=19.8 only done in the best seeing (20 nights/yr) pixel size is 0.4 arcsec, astrometric precision is 60 milliarcsec Southern Galactic Cap multiple scans (> 30 times) of the same stripe Continuous data rate of 8 Mbytes/sec Alex Szalay, JHU

12. Survey Strategy Overlapping 2.5 degree wide stripes Avoiding the Galactic Plane (dust) Multiple exposures on the three Southern stripes Alex Szalay, JHU

13. The Spectroscopic Survey Measure redshifts of objects  distance SDSS Redshift Survey: 1 million galaxies 100,000 quasars 100,000 stars Two high throughput spectrographs spectral range 3900-9200 Å. 640 spectra simultaneously. R=2000 resolution. Automated reduction of spectra Very high sampling density and completeness Objects in other catalogs also targeted Alex Szalay, JHU

14. The Mosaic Camera Alex Szalay, JHU

15. First Light Images Telescope: First light May 9th 1998 Equatorial scans Alex Szalay, JHU

16. The First Stripes Camera: 5 color imaging of >100 square degrees Multiple scans across the same fields Photometric limits as expected Alex Szalay, JHU

17. NGC 2068 Alex Szalay, JHU

18. UGC 3214 Alex Szalay, JHU

19. NGC 6070 Alex Szalay, JHU

20. The First Quasars The four highest redshift quasars have been found in the first SDSS test data ! Alex Szalay, JHU

21. SDSS T-dwarf (June 1999) Methane/T Dwarf Discovery of several newobjects by SDSS & 2MASS Alex Szalay, JHU

22. Detection of Gravitational Lensing 28,000 foreground galaxies and 2,045,000 background galaxies in test data(McKay etal 1999) Alex Szalay, JHU

23. SDSS Data Flow Alex Szalay, JHU

24. Data Processing Pipelines Alex Szalay, JHU

25. Other Archives Other Archives Other Archives Concept of the SDSS Archive Science Archive (products accessible to users) OperationalArchive (raw + processed data) Alex Szalay, JHU

26. Distributed Collaboration Fermilab U.Chicago U.Washington ESNET I. AdvancedStudy Japan Princeton U. VBNS JHU Apache PointObservatory NMSU USNO Alex Szalay, JHU

27. SDSS Data Products Object catalog 400 GB parameters of >108 objects Redshift Catalog 1 GB parameters of 106 objects Atlas Images 1.5 TB 5 color cutouts of >108 objects Spectra 60 GB in a one-dimensional form Derived Catalogs 20 GB - clusters - QSO absorption lines 4x4 Pixel All-Sky Map 60 GB heavily compressed All raw data saved in a tape vault at Fermilab Alex Szalay, JHU

28. Attributes Number Sky Position 3 Multiband Fluxes N = 5+ Other M= 100+ Geometric Indexing “Divide and Conquer” Partitioning 3NM HierarchicalTriangular Mesh Split as k-d treeStored as r-treeof bounding boxes Using regularindexing techniques Alex Szalay, JHU

29. Distributed Implementation User Interface Analysis Engine Master SX Engine Objectivity Federation Objectivity Slave Slave Slave Objectivity Slave Objectivity Objectivity RAID Objectivity RAID RAID RAID Alex Szalay, JHU

30. Collaboration with Particle Physics Collaboration with the Analysis Data Grid: proposal to the NSF KDI program by JHU, Fermilab and Caltech (H. Newman, J. Bunn) + Objectivity, Intel and Microsoft (Jim Gray) Involves computer scientists, astronomers and particle physicists Accessing Large Distributed Archives in Astronomy and Particle Physics experiment with scalable server architectures, create middleware of intelligent query agents, apply to both particle physics and astrophysics data sets Status: 3 year proposal just funded Alex Szalay, JHU

31. The Age of Mega-Surveys The next generation of astronomical archives with Terabyte catalogs will dramatically change astronomy top-down design large sky coverage built on sound statistical plans uniform, homogeneous, well calibrated well controlled and documented systematics The technology to acquire, store and index the data is here we are riding Moore’s Law Data mining in such vast archives will be a challenge, but possibilities are quite unimaginable Integrating these archives into a single entity is a project for the whole community=> Virtual National Observatory Alex Szalay, JHU

32. New Astronomy – Different! Systematic Data Exploration will have a central role in the New Astronomy Digital Archives of the Sky will be the main access to data Data “Avalanche” the flood of Terabytes of data is already happening, whether we like it or not! Transition to the new may be organized or chaotic Alex Szalay, JHU

33. NVO: The Challenges Size of the archived data • 40,000 square degrees is 2 trillion pixels • One band: 4 Terabytes • Multi-wavelength: 10-100 Terabytes • Time dimension: few Petabytes The development of • new archival methods • new analysis tools • new standards (metadata, interchange formats) Hardware/networking requirements Training the next generation! Alex Szalay, JHU

34. Summary The SDSS project combines astronomy, physics, and computer science It promises to fundamentally change our view of the universe It will determine how the largest structures in the universe were formed It will serve as the standard astronomy reference for several decades Its ‘virtual universe’ can be explored by both scientists and the public Through its archive it will create a new paradigm in astronomy Alex Szalay, JHU

35. Alex Szalay, JHU