1 / 37

Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal

Edge Services Framework ESF in Open Science Grid. Joint OSG and EGEE Operations Workshop September 27-29, 2005 RAL, UK. Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal.gov. Kate Keahey Argonne Lab | Globus Alliance keahey@mcs.anl.gov. Outline. OSG-ESF Activity. ESF Phase 1.

garin
Download Presentation

Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Edge Services FrameworkESF in Open Science Grid Joint OSG and EGEE Operations Workshop September 27-29, 2005 RAL, UK Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal.gov Kate Keahey Argonne Lab | Globus Alliance keahey@mcs.anl.gov

  2. Outline • OSG-ESF Activity. • ESF Phase 1. • Concepts and Design. • ESF future direction. • Xen overview. • Plan for Phase 1 on OSG. • Site admin responsibilities. • VO admin responsibilities. • Status and Next Steps.

  3. Vision From talk by Frank Wuerthwein at OSG Consortium Meeting 2005 (simplified snapshot of a typical OSG/LCG site in 2007)

  4. OSG-ESF Activity • Started in September 2005. • Physicists, Computer Scientists & Engineers, Software Architects. • Chairs: Kate Keahey and Abhishek Singh Rana. • Workspace Services Design: • Timothy Freeman & Kate Keahey (Globus Alliance). • Edge Services Implementations: • USATLAS: Rob Gardner (U Chicago), Alexander Vaniachine (ANL). • USCMS: Frank Wuerthwein, Abhishek Singh Rana (UC San Diego). • Mailing List – osg-edgeservices at opensciencegrid.org • Web collaborative area – http://osg.ivdgl.org/twiki/bin/view/EdgeServices

  5. Current OSG-ESF mailing list

  6. ESF - Phase 1

  7. No ESF - Phase 0 CE SE Site

  8. No ESF - Phase 0 Static deployment CE CMS ATLAS CDF SE Site

  9. ESF? CE SE Site

  10. ESF - Phase 1 Snapshot of ES Wafers implemented as Virtual Workspaces ESF ATLAS CMS CE SE CDF Guest VO Site

  11. An attempt at ESF Terminology • Edge Services Wafer (ES Wafer) • A specific instance of a dynamically-created VM (workspace) is called an Edge Services Wafer. • An ES Wafer can have several Edge Services running. • A VO can have multiple ES Wafers up at a Site. • Edge Services Slot (ES Slot) • An ES Slot has hardware characteristics specified by the Site Admin. • An ES Slot can be leased by a VO to host an ES Wafer. • Edge Service (ES) • A VO-specific service instantiated by a VO in a Wafer. • Workspace Service (WS) • Service at a Site that allows VOs to instantiate ES Wafers in ES Slots.

  12. ESF - Phase 1 Snapshot of ES Wafers implemented as Virtual Workspaces Dynamically deployed ES Wafers for each VO GT4 Workspace Service & VMM ESF Wafer images stored in SE ATLAS CMS CE SE CDF Guest VO Site Compute nodes and Storage nodes

  13. User jobs at Compute nodes using ES Wafers for VO Edge Services ESF ATLAS CMS CE SE CDF Guest VO Site

  14. VO Admin transporting/storing ES image to a remote Site....Deploying ES using image stored in Site’s local repository

  15. ESF - Phase 1 Role=VO Admin CMS ESF CE SE Site

  16. ESF - Phase 1 Role=VO Admin CMS ESF CE SE Site

  17. ESF - Phase 1 Role=VO Admin ESF CE SE Site

  18. ESF - Phase 1 Role=VO Admin ESF CE SE Site

  19. ESF - Phase 1 Role=VO Admin ESF CMS CE SE Site

  20. ESF - Phase 1 Role=VO Admin ESF CMS CE SE Site

  21. ESF - Phase 1 Role=VO Admin ESF CMS CE SE Site

  22. A VO User using ES..

  23. ESF - Phase 1 Role=VO User ESF CMS CE SE Site

  24. ESF - Phase 1 Role=VO User ESF CMS CE SE Site

  25. ESF - Phase 1 Role=VO User ESF CMS CE SE Site

  26. ESF - Phase 1 Role=VO User ESF CMS CE SE Site

  27. ESF - future direction

  28. ESF - future direction • Same concept. • ES Wafers to be deployed as an ESF cluster and be fully schedulable by any OSG VO.

  29. ESF - future direction ESF Dynamically deployed ES Wafers for many VOs Brokering and Scheduling by Edge Services Framework ATLAS2 CMS ATLAS1 CDF Cluster of ES Slots with different properties Site

  30. Xen overview Public Network Private Network Virtual Machine Monitor (VMM) Hardware

  31. Plan for Phase 1 on OSG

  32. Plan for Phase 1 on OSG • ATLAS & CMS procure one piece of hardware at their Sites on OSG that runs ESF (called ESF node). • Dual CPU. • 4GB RAM (No PAE support in Xen2, we will exploit only 4GB until we migrate to Xen3). • Site administrators install: • Xen (We will go for Xen 2.0.7). • GT4 Workspace Service. • VO administrators use ESF to fire-up Xen VMs that instantiate VO Services – Edge Services in an ES Wafer. • A single ESF node hosts ES Wafers for both ATLAS & CMS.

  33. Site Administrator Responsibilities • Start with FC4 and deploy: • Xen (We will go for Xen 2.0.7). • Custom kernel for domain 0 (grub required). • Custom kernel for domain U. • GT4. • GT4 Workspace Service. • Provision: • One public IP, One private IP per VM. • Host certificates per VM. • Disk space per VM. • Declare available ES Slots and their properties to ESF.

  34. VO Administrator Responsibilities • Prepare (freeze) ES Wafer instance as disk image. • Prepare image into file for transport. • SRMCP the image to remote Site’s SE. • Use ESF to fire-up Xen VM with VO-Services (ES Wafer) at remote Site, from image file in remote SE, using role based authorization (OSG-RBAC). • Advertise the running Edge Services as needed.

  35. Status • Xen3 on testbed. • GT4 Workspace Service being used. • Prototype 0 of ESF. Next Steps • Revert back to Xen2. • Develop VO-Service images. • Verify performance, functionality, robustness.

  36. Thank You.

More Related