1 / 12

The Astro-Wise Federation

The Astro-Wise Federation. … staying connected Lorentz center, Monday, 31 March 2008. The components of the federation. Databases Identical copies at all AstroWise nodes Dataservers Multiple dataservers per node Cache frequently accessed data from other nodes

kerri
Download Presentation

The Astro-Wise Federation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Astro-Wise Federation … staying connected Lorentz center, Monday, 31 March 2008

  2. The components of the federation • Databases • Identical copies at all AstroWise nodes • Dataservers • Multiple dataservers per node • Cache frequently accessed data from other nodes • DPUs – Distributed Processing Units • Process data using database and dataservers at the “local” node • Networks • Users

  3. Databases • Hub-and-spoke replication using Oracle Streams • Groningen as hub • Bonn, München, Napoli as spokes • Databases can be used locally if network to hub is down • Active since about one year

  4. Databases • 15 GByte metadata • 60 GByte sources and associations • excluding USNO and SDSS • 50 GByte indexes • 600 GByte filespace • including USNO and SDSS

  5. Databases • Groningen 581 GByte • Includes SDSS, USNO • Bonn 1 GByte • Copying from scratch after crash • München 70 GByte • Raw FITS headers missing • Napoli 7 GByte • Sources and Associations still missing

  6. Databases – what next • Upgrade database hardware • Groningen • Upgrade to Oracle 11g • Groningen, München • Increase network bandwidth • Napoli • Copy remaining data • Local accounts for federated users

  7. Dataservers • Peer-to-peer • Write locally, read anywhere • Local copy in cache of remote files • Connected for more than a year • Today connected between Bonn, Groningen, München, Napoli and Nijmegen

  8. Dataservers • 25 Dataservers • 363 TByte total online storage • 23 TByte in use • 2 million files

  9. DPUs • Distributed Processing Unit • Clusters with 8 nodes up to 100’s of nodes • Front end to GRID processing • Active for more than a year • Available in Bonn, Groningen, München, Napoli and soon in Nijmegen

  10. Networks • Speed from or to Groningen • Bonn 2~5 MByte/s • München 2~5 MByte/s • Napoli <0.5 MByte/s • 5MByte/s radio link to be installed • Nijmegen >5 MByte/s ? • Databases and dataservers share the same bandwidth

  11. Discussion • Questions? • Comments? • Suggestions?

More Related