1 / 14

Astro-Wise and EGEE

Astro-Wise and EGEE. Fokke Dijkstra Donald Smits Center for Information Technology University of Groningen, NL In collaboration with OmegaCEN, Kapteyn institute, University of Groningen, NL 23 September - EGEE conference Istanbul. Introduction. Astro-Wise setup. TARGET diagram.

hea
Download Presentation

Astro-Wise and EGEE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Astro-Wise and EGEE Fokke Dijkstra Donald Smits Center for Information Technology University of Groningen, NL In collaboration with OmegaCEN, Kapteyn institute, University of Groningen, NL 23 September - EGEE conference Istanbul

  2. Introduction

  3. Astro-Wise setup

  4. TARGET diagram Extreme data lineage!

  5. Astro-WISE: Services

  6. Lofar in short • Large distributed radio telescope • In NL 2008: initial roll out 20 stations • Full scale aperture synthesis array, extends 100 km • Fibre network, Software Correlator • Run as a broad common-user observatory • Making the transition now from“project” to “observatory” • Extended with European partners • Up to 4 Petabytes/year to archive

  7. New Tier 1 location Jülich - Germany Code base Amsterdam CIT / Tape Meta data and Files CIT – Groningen Blue Gene Nearline storage & processing Global FS view Astro-WISE SAS Archive Oracle DB Export by file C=IT/O=INFN /L=CNAF/CN=Pinco Palla/CN=proxy C=IT/O=INFN /L=CNAF/CN=Pinco Palla/CN=proxy Tier 0 Astro-WISE view Tier 1 VOMSpseudo-cert VOMSpseudo-cert Communities - Groningen - Amsterdam - Leiden - Nijmegen - Jülich Euro VO Lofar Communities Grid user EGEE Tier 2 LoWise SRM

  8. Astro-Wise and Grid • User authentication • VOMS proxies • Proxy server • Preferably client installation • Python only! • Web portal (should work) • Astro-Wise system will obtain delegated proxy • Extend to Oracle database • Job submission • Manage collection of tasks • Storage • Support SRM

  9. connects connects tasks Linux cluster tasks connects credentials VOMS server Proxy server Astro-Wise Database DPU VOMSattributes tasks credentials Astro-Wise job submission Astro-Wise user gLite UI

  10. Problems encountered • Preferrably no gLite-UI installation for all users • Currently only myproxy-init really necessary • Python implementation needed • OMII-UK (NDG) seems to have one, earlier tests unsuccessful • Other option java webstart version • CCLRC version works

  11. Job submission • Image processing inherently parallel • e.g. Processing per single CCD image • One step requires global communication • Communication through database • MPI job type used for processing • No real MPI program used • Independent processes started using ssh

  12. Future plans • Investigate current state of python myproxy implementation as used by NDG • Use of SRM for Lofar storage (StoRM on global file system) • Further integration of GSI with Astro-Wise and Oracle • Allow Astro-Wise to store data through SRM

  13. Final remarks • Grid job submission relatively easy • Credential management stumble block for users • Obtaining certificate • Managing (proxy) certificates • Renewing certificates • Lack of portable tools • Cannot ship current gLite UI to everyone • UI portals may be a solution • Advantages of Grid • Standards! • Job submission • Data management • Security

  14. Acknowledgments • Kor Begeman (implementation in Astro-Wise) • Willem Jan Vriend (implementation in Astro-Wise) • Danny Boxhoorn • Andrey Belikov All from: OmegaCEN Kapteyn Institute University of Groningen, NL

More Related