1 / 24

Millennium Data Dissemination

Millennium Data Dissemination. databases and other services. FOF groups and Subhalos. Mock images. Raw data: Particles. Density fields. Subhalo merger trees. Synthetic galaxies (SAM). Mock catalogues. MRObs. Scaling . Simulation Particles . FOF Groups. (Sub-) Halos.

hedia
Download Presentation

Millennium Data Dissemination

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Millennium Data Dissemination databases and other services Millennium Workshop

  2. FOF groups and Subhalos Mock images Raw data: Particles Density fields Subhalo merger trees Synthetic galaxies (SAM) Mock catalogues Millennium Workshop

  3. MRObs Scaling Simulation Particles FOF Groups (Sub-)Halos Halo Merger Trees FOF Group Finder SUBFIND Tree Builder Gadget SAM SSP SExtractor SkyMaker’ MoMaF’ SAM Galaxies SExtractor Catalogues Synthetic Images Light-cone Galaxies IGM PSF, Noise, Background Millennium Workshop

  4. Virgo-Millennium Database • Online access to Millennium simulations etal • portal: http://www/mpa-garching.mpg.de/millennium • Databases • public: http://gavo.mpa-garching.mpg.de/Millennium • private + MyDB: http://gavo.mpa-garching.mpg.de/MyMillennium • Improved database access • MillenniumTAP batch queries • http://galformod.mpa-garching.mpg.de/millenniumtap Millennium Workshop

  5. Millennium data model (old) 5

  6. Trees in Database last progenitor main leaf and root Millennium Workshop

  7. Merger trees particles halos Millennium Workshop

  8. Joining Garching and Durham 8

  9. Millennium Workshop

  10. Most recent data sets(@MPA) • New simulations: • MR-XXL: Anguloetal 2012 • Millennium WMAP7 (Thomas etal, MilliGas?) • Updated semi-analytics, with (scaling to) WMAP7, MCMC • Guo etal 2011, 2012 • Henriques etal 2012/3 • New synthetic light cone catalogues • Henriques etal 2012 • also some all-sky(SDSS depth) • Millennium Run Observatory • Overzier etal2012, images+Sextractor catalogues • Most loaded in new databases • soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) • New DB functions: • spatial queries • possibly: scaling, … Millennium Workshop

  11. AllSky(Paritosh Verma) “All” smoothed “All” “Red” “Blue” Millennium Workshop

  12. Most recent data sets(@MPA) • New simulations: • MR-XXL: Anguloetal 2012 • Millennium WMAP7 (Thomas etal, MilliGas?) • Updated semi-analytics, with (scaling to) WMAP7, MCMC • Guo etal 2011, 2012 • Henriques etal 2012/3 • New synthetic light cone catalogues • Henriques etal 2012 • also some all-sky(SDSS depth) • Millennium Run Observatory • Overzier etal2012, images+Sextractor catalogues • Most loaded in new databases • soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) • New DB functions: • spatial queries • possibly: scaling, … Millennium Workshop

  13. Spatial queries in the databaseSpatial3Dlibrary (with Tamas Budavari) • Searches for objects in spatial subsets of simulation box is hard • C# library for optimising such spatial queries using PH space filling curve • Soon in Millennium DBs (needs documentation) Millennium Workshop

  14. with cover as ( -- some details left out select * from dbo.fSimulationCoverSphere( Simulation::Millennium, Sphere3D::newInstance(0,0,0,20),6) ) select f.* from cover c , millennium..fof f where f.snapnum=67 and f.phkey between c.keymin and keymax and c.shape.ContainsPoint(f.x, f.y, f.z)=1

  15. Query shape syntax • Primitives : • box[ 0,0,0,10,10,10 ], sphere[10,10,10,20] • cone[0,0,0,1,1,1,.1,600] vertex,direction,angle,min-depth,max-depth, • conesegment[0,0,0,1,1,1,.1,500,600]vertex,direction,angle,min-depth,max-depth • Combinations : • union/intersection/difference [ shape (, shape)*]

  16. used in MXXL browser courtesy RaulAngulo

  17. In the future: light cones in SQL(requires galaxy catalogues with dM/dz) Millennium Workshop

  18. Usage statistics • 589 registered users • > 17.3 million queries (7532/day) • > 287 billion rows (137 million/day) queries,rows,users Millennium Workshop

  19. Distributions over users Millennium Workshop

  20. Millennium TAP • New DB access service • All MyMillennium users have access • Batch queue • longer timeout • capability to kill queries • history • Implements IVOA’s Table Access Protocol (TAP) • AJAX frontend: TAPFace • TAP aware tools can query it, TOPCAT • Some differences • schema vs database (Non SQLServer DBs can be served) • no mydb (yet) • Demo this afternoon Millennium Workshop

  21. Warning It’s not always as simple as it seems.One should try to understand the data when using the database thanks toKevin Bundy, TommasoTreu, Richard Ellis http://astro.berkeley.edu/~kbundy/millennium/ Millennium Workshop

  22. Other Millennium data dissemination services • portal: http://galformod.mpa-garching.mpg.de • Simulation service(s) • SeSam online (Neistein, Weinmann , Bourges) • L-Galaxiesonline (see talk by Matthias Egger) • Image Browsers: • may help define where to point your virtual telescope. • MR-XXL browser (public | interactive) (Angulo), • MRObs image browser (public | private) (Overzier etal) Millennium Workshop

  23. Service vs other data sets • Databases • HydroClustersdatabase (Dolag) • Virtual telescopes • Smac (Dolag, Reynecke, Bourges) • PHOX(Biffietal) • Image Browsers: • Phoxy browser + PHOX Millennium Workshop

  24. Thank you. 2012-12-17 Millennium Workshop Millennium Workshop 24

More Related