1 / 9

Features of the SDSS

Features of the SDSS. Special 2.5m telescope, at Apache Point, NM 3 degree field of view Zero distortion focal plane Two surveys in one Photometric survey in 5 bands - 200 million objects Spectroscopic redshift survey - 1 million distances Automated data reduction

leo-harris
Download Presentation

Features of the SDSS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Features of the SDSS Special 2.5m telescope, at Apache Point, NM 3 degree field of view Zero distortion focal plane Two surveys in one Photometric survey in 5 bands - 200 million objects Spectroscopic redshift survey - 1 million distances Automated data reduction Over 120 man-years of development (Fermilab + collaboration scientists) Very high data volume Expect over 40 TB of raw data About 2 TB processed catalogs Data made available to the public

  2. Data Processing Pipelines

  3. SDSS Data Products Object catalog 500 GBparameters of >108 objects Redshift Catalog 1 GBparameters of 106 objects Atlas Images 1500 GB5 color cutouts of >108 objects Spectra 60 GB in a one-dimensional form Derived Catalogs 20 GB clusters QSO absorption lines 4x4 Pixel All-Sky Map 60 GB heavily compressed Corrected Frames 15 TB All raw data (40TB) saved at Fermilab

  4. Accessing the Data • Few fixed access patterns • one cannot build indices for all possible queries • worst case scenario is linear scan of the whole table • Increasingly large differences between • Random access • Sequential I/O • Often much faster to scan than to seek • Good layout of data => more sequential I/O • Geometric indexing – partitioning in storage • Using Objectivity/DB • Ported to MS SQL Server (w. Jim Gray)

  5. SDSS in GriPhyN • Two Tier2 Nodes (FNAL+JHU) • testing framework on real data in different scenarios • FNAL node • massive reprocessing of images • full regeneration of catalogs from the images (on disk) • gravitational lensing, finer morphological classification • Image coaddition, differencing • JHU node • catalog calculations, integrated with database • tasks require lots of data, can be run in parallel • various statistical calculations, likelihood analyses • power spectra, correlation functions, Monte-Carlo • Public access • creating virtual data for NVO services (implemented later)

  6. The SDSS Southern Survey • Scanning a single stripe on the sky >30 times over • Coaddition => extra depth • Differencing => time dimension • Multiple ways to combine the stripes • Rerun the pipelines with custom parameters • Build a new object catalog • Perform particular science analysis (lensing map) • On the right timescale to try GriPhyN framework

  7. Large Scale Statistical Analysis • Galaxy distribution has non-trivial clustering patterns • Reflects conditions in the early universe • Spatial statistical tools to be run on object catalog, applying many different cuts to the data • Spatial power spectrum • Correlation functions • These algorithms are typically N2 or N3 with the number objects!! • Some of the analyses will partition well (likelihood), others will not (pair counts)

  8. Trends in Astronomy Future dominated by detector improvements • Moore’s Law growth in CCD capabilities • Gigapixel arrays on the horizon • Improvements in computing and storage will track growth in data volume • Investment in software is critical, and growing Total area of 3m+ telescopes in the world in m2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels.

  9. VO- The challenges • Large number of new surveys • multi-TB in size, 100 million objects or more • individual archives planned, or under way • Multi-wavelength view of the sky • more than 13 wavelength coverage in 5 years • Size of the archived data 40,000 square degrees is 2 Trillion pixels • One band 4 Terabytes • Multi-wavelength 10-100 Terabytes • Time dimension 10 Petabytes • Current techniques inadequate • Scalable hardware/networking requirements • Transition to the new astronomy MACHO 2MASS DENIS SDSS DPOSS GSC-II VISTA COBE MAP NVSS FIRST GALEX ROSAT OGLE, ...

More Related