1 / 70

Grid Astronomy with Image Federation Roy Williams Michael Feldmann

Grid Astronomy with Image Federation Roy Williams Michael Feldmann California Institute of Technology. This Presentation. Why (Roy Williams) Image Federation Standard protocols, standard data types Grid Services: eg. Simple Image Access (SIA) Image Mosaicking with Atlasmaker

sheena
Download Presentation

Grid Astronomy with Image Federation Roy Williams Michael Feldmann

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Astronomy with Image Federation Roy Williams Michael Feldmann California Institute of Technology

  2. This Presentation Why (Roy Williams) • Image Federation • Standard protocols, standard data types • Grid Services: eg. Simple Image Access (SIA) • Image Mosaicking with Atlasmaker How (Mike Feldmann) • Condor • Queues • SRB • ????

  3. 1. NVO Image Service: Federation Needs Standards

  4. NVO Image ProtocolSIAP • Specify box by position and size • SIAP server returns relevant images • Footprint • Logical Name • URL Can choose: standard URL: http://....... SRB URL srb://nvo.npaci.edu/…..

  5. Simple Image Access Service • Query is sky region • May query on image type, image geometry • Response is VOTable of images • Each has WCS (geometry) parameters • Plus a URL to fetch the image • Designed for • Set of pointed observations (eg Hubble) • Wide-area survey (eg Sloan) • Image service • Mosaicking • Reprojection

  6. 2. Atlasmaker: Grid-based Image Federation

  7. Image Federation

  8. Multispectral Imagery Crab Nebula.3 channels: X-ray in blue, optical in green, and radio in red. Moffet Field California. 224 channels from 400 nm to 2500 nm

  9. Images of the same galaxy taken several days apart are automatically subtracted from one another, and remaining bright spots may be supernova candidates. (NEAT project) detection Image subtraction allows detection of narrow-line features that are not also wide-band (eg Hα but not R-band) Image Federation Stacking allows detection of faint sources. A 1-sigma detection in each of many bands becomes a 3-sigma detection. It's A New Window!

  10. Multi-Wavelength Image Morphology DPOSS-2MASS Image Mosaics J F N J H K J F N J H K Galaxy identifcation, galaxy clusters Pattern matching with shape AND color

  11. Virtual Skyhttp://virtualsky.org Galaxy cluster in Xray, DPOSS, SloanThis is M77 & has Xray and Radio tooComa

  12. Virtual Sky: Image Federation http://virtualsky.org/ from Caltech CACR Caltech Astronomy Microsoft Research Virtual Sky has 14,000,000 tiles 140 Gbyte Change scale Change theme Optical (DPOSS) Xray (ROSAT) theme Coma cluster

  13. Image Federation

  14. Virtual Sky Roy Williams, Caltech CACR Alex Szalay, Johns Hopkins University Ashish Mahabal, Caltech Astronomy Jim Gray, Microsoft Research George Djorgovski, Caltech Astronomy Julian Bunn, Caltech CACR Robert Brunner, Caltech Astronomy

  15. Principle components – Information concentration Multichannel Images L is 23 cm wavelength C is 10 cm wavelength H is horizontal polarization V is vertical polarization A color image is 3 channels

  16. Principle Components SDSS (5 channel) SDSS+2MASS (8 channel)

  17. Mosaicking and Federation Infrared map Mosaicking • Every Astronomical image has a different projection • different pointing of the telescope • We want to mosaic different images • We want to federate different information • Compute intensive: • flux in each pixel is carefully distributed into a new pixel grid Xray map today Federation Xray map last year

  18. AtlasmakerUses Montage, Yoursky Project Estimate & correct Background Co-Add Project David Hockney Pearblossom Highway 1986 Data Chart

  19. Background Correction Uncorrected Corrected

  20. Montage Background Correction Project pixels to output chart Fit ramps on overlap regions Fit ramps on projected images Subtract from Pixel values

  21. Images and Charts • Image • Big data • Chart • Map: sphere → plane • FITS-WCS header • small data An atlas is a collection of charts Hyperatlas is an attempt to standardize atlases

  22. Vchart TM-5-SIN-20-1589 TAN projection SIN projection Hyperatlas Standard naming for atlases and vcharts TM-5-SIN-20 Standard Scales: scale s means 220-s arcseconds per pixel Standard Layout TM-5 layout Standard Projections HV-4 layout

  23. Charts and Pages 2. Page – a sized chart chosen from a standard set – an Atlas 1. Chart – a metadata entity specifying a map from sphere to plane 3. A Tiling of a Page SIN projection The virtual disk is 400,000 pixels wide

  24. Over to Mike

  25. Neighboring Atlasmaker tile Survey images overlap multiple tiles Atlasmaker DAG (tile level) Atlasmaker tile Survey images An Atlasmaker tile is some region of the sky. Can retrieve needed images for a given page with SIAP services. Images returned by SIAP server for a page query establish dependency graph for the construction of this page. A tile is simply a subset of this page.

  26. Atlasmaker DAG (tile to page) • An Atlasmaker “page” is made up of many smaller “tiles”. • The page defines the “projection” (Montage) • A tile is a simple unit of work to be completed (easy to deal with computationally) Page Many tiles

  27. Tile A Tile B survey images Atlasmaker DAG (tile level) Raw images on page border must be projected twice. Images on the border of two tiles of the same page do not need to be projected twice (this efficiency requires image product archival and good bookkeeping).

  28. ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... Atlasmaker DAG (page and up) coarse resolution Web based navigation tool many levels resolution page high integrity images compressed images (page-level DAG)

  29. Image compression Reduce resolution Projection defined Montage SIAP web service retrieval establishes dependencies Raw images archived Large dataset management (SRB) Atlasmaker DAG Coarse resolution Web based navigation tool Pages Tiles Projected Images Raw Images

  30. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  31. Atlasmaker DAG Construction Probe top compressed Image Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  32. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  33. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  34. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  35. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  36. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  37. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  38. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  39. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  40. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  41. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  42. Atlasmaker DAG Construction Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  43. Atlasmaker DAG Construction Probe all compressed Images Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  44. Atlasmaker DAG Construction Complete!!! Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

  45. Something breaks!!! • Some survey images might have been flawed • A valid state might have been set by a failed computation • New/better images now exist • We do not want to recalculate entire atlas!

  46. Why Virtual Data? Oops!

  47. Atlasmaker DAG Maintenance Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state Discover an invalid state!!

  48. Atlasmaker DAG Maintenance Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state Propagate invalid state

  49. Atlasmaker DAG Maintenance Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state Fix invalid raw image

  50. Atlasmaker DAG Maintenance Probe all compressed Images that are not in a “valid” state Course resolution Pages Tiles Projected Images Initial state Invalid state Working state Raw Images Valid state

More Related