1 / 42

Planning

Planning. Hui Li Sep 13, 2004. Why another “Grid”?. Grid is Learn the latest Grid technologies by building a Campus Grid A system that works: Harnessing the idle cycles of desktops Intra and inter faculty resource sharing Have fun. HOT. The Evolution of Species - Sharing.

lael
Download Presentation

Planning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Planning Hui Li Sep 13, 2004

  2. Why another “Grid”? • Grid is • Learn the latest Grid technologies by building a Campus Grid • A system that works: • Harnessing the idle cycles of desktops • Intra and inter faculty resource sharing • Have fun HOT

  3. The Evolution of Species - Sharing Resource Sharing: P2P, Web Services, Grids Information Sharing: WWW, HTTP, HTML Communications and Data Sharing: Email, ftp, telnet, TCP/IP Sharing Networking ARPANET

  4. Levels of Resource Sharing Cluster Grid Campus Grid Global Grid

  5. Enabling Technologies • The key is middleware: virtualizing and coordinating the underlying resources • Open source

  6. Campus Grid Worldwide • Nanyang Technological University (NTU) • University of Houston • University of Wisconsin at Madison • Virginia Tech (VT) • Tokyo Institute of Technology (TiTech) • University of Texas at Austin

  7. NTU Campus Grid • Partners: CSV, BIRC, PDCC, APSTC, SUN • Globus, SGE, Ganglia, MapCenter, ClearningHouse portal

  8. University of Houston • Partners: CS, HPC group • EZ-Grid (resource brokering)

  9. University of Wisconsin at Madison • Condor, Virtual Data Toolkit (VDT)

  10. Virginia Tech • Teracluster: 1,100 dual processor 2Ghz PowerMac G5.

  11. Tokyo Institute of Technology

  12. UT/TACC Grid • Alliance with IBM to build the largest Campus Grid in US • Support for a large community of scientific researchers (Engineering, Biology, Physics, Chemistry, Computer Science, Geosciences, Arts, Business, etc) • Middleware: GridPort

  13. Summary of Campus Grid Projects • Have a couple of successful cases but still in the early stage of development • Driven by the growing need for computing power of campus researchers • Driven and enabled by the emerging Grid research and technologies • Funding and industrial support

  14. them us Desktops in LAN DAS-2 cluster (as users) • ~ 200 Pentium III 1 GHz nodes • 64MB RAM • 20 GB local disk • Fast Ethernet interface • Myrinet interface Our Situation DAS cluster in Leiden (full control) • ~ 20 Pentium Pro 200MHz nodes • 64MB RAM • 2.5 GB local disk • Fast Ethernet interface • Utilizing the existing IT infrastructure AMAP • Harnessing idle cycles • Demonstrate that a group of students can actually build a functional system

  15. Small Fish’s Approach Planning (first phase): • Building the testbed • Deploying middleware • Developing high-level services • Proof-of-concept applications • Building and sharing expertise • Prepare going campus-wide

  16. Applications Portal (web interface, user support) RB & WM IS & Monitoring Data Mgr. Local RM Security Fabric (testbed, OS) Big Picture

  17. Project 1: Fabric Management - Setting Up the Testbed • Networking (DHCP, NFS, NTP, DNS, etc) • Linux Automatic Installation Infrastructure • Middleware selection, installation and configuration (VDT) • Security Infrastructure (NIKHEF CA) • Testing and verification • User Support: web site, documentation, mailing lists, trouble ticket system (Bugzilla)

  18. Virtual Data Toolkit (VDT) • Globus • Condor/Condor-G • Virtual Data System (Chimera, etc) • MyProxy • MonaLisa • Netlogger • Various patches, packages, and utilities

  19. Project 2: Stealing the Idle Cycles

  20. Project 2: Stealing the Idle Cycles • Condor: open source, widely deployed, multi-platform support, etc • Setup: a linux Condor pool, several Windoz XP/NT machines with Condor packages. • Goal: Investigate the feasibility, policy, and configuration of desktop computing with Condor.

  21. Project 3: LUCGrid Portal • A Grid portal is a web server that provides an interface to Grid services, allowing users to submit jobs, transfer files, and query Grid information services from a standard web browser. • Tools to facilitate building portals: • OGCE (Open Grid Computing Environment) • Goal: • Setting up a portal server (lucgrid.liacs.nl) • Integrating with other Grid components

  22. Information Services Component (Proj. 5) Resource Management Component (Proj. 4) Portal (Proj. 3) Security and VO Component (Proj. 6) Applications Component (Proj. 8) Project Integration Data Management Component (Proj. 7)

  23. Project 4: Resource Brokering and Workload Management

  24. Project 4: Resource Brokering and Workload Management • Deployment • Job submission interfaces to portals • Interfaces to local resource management systems (PBS, Condor, etc) • Interfaces to Information Services • Development • Simple brokering services

  25. Project 5: Information Services and monitoring

  26. Project 5: Information Services and monitoring • Cluster monitoring using NWS, Ganglia, MapCenter, and MonaLisa • Interfaces to Portals • Interfaces to RM components • Interfaces to DM components • (If interested): Pull-based on-demand information framework (Surfer, etc)

  27. Project 6: Security and VO Management • Help with authentication infrastructure in Project 1 (fabric management) • MyProxy - integration with portal • VO (Virtual Organization) Management • Setting up multiple VOs • Authorization and Manage VOs • VOMS software

  28. Project 7: Data Management • Interfaces to portals • Data transfer using GridFTP • Network monitoring with Netlogger • Virtual Data System • Replica location and selection service • Demo: Transfer large files between two sites through the portal

  29. Project 8: Applications (Proof-of-Concept) • Adapting applications to run on a Grid • Potential application domains: Bioinformatics (BLAST), Medicine, Physics, Computational Sciences, etc • Goals: • Demo applications through the portal • Documentation (quick-start guide, user’s guide) • Communications with other faculties

  30. Project Summary • A functional Grid testbed with: software stack, portal interfaces, demo applications, and user support • Final report • Detailed documentation of what you have done • Submission to portal for dissemination • Research projects and opportunities

  31. Project Topics • System - LUCGrid (FCFS, 7) • Fabric Management (1) • LUCGrid Portal (1) • Desktop Computing (1) • Resource Management (1) • Information Services and Monitoring (1) • Security (1) • Data Management (1) • Applications (Flexible, e.g. Bioinformatics, Finance, CS) • Others • OGSA and WSRF (Web Services) • P2P • Semantic Web • … …

  32. Project Organization (System Track) • Coordinators - Hui and Lex, Advisor - David • Nov 8 - Planning and starting LUCGrid project • Nov 15, 22 and 29 - Deployment by individual projects (coordinated by Hui, fabric, security and portal students) • Dec 6 - Integration meeting and retrospect • Final report and demo

  33. Project Organization (Application and others) • Same as previous years • Implement an grid-enabled application that runs on multiple clusters (DAS-2) • Flexible in area selection • For other possibilities (Web Services, P2P, etc), discuss with us

  34. Presentation Topics • Sep 27 - Fabric Management / Resource Management • Oct 11 - Information Services / Security • Oct 18 - Data Management / Networking • Oct 25 - Grid Portals / Applications (User Cases) • Nov 1 - OGSA and WSRF / P2P Computing

  35. Presentation Organizations • Two topics every time • Two students every topic (45 minutes) • Prepare one presentation together • 1st student: intro, architecture (~20 min) • 2nd student: software, system (~20 min) • Questions and discussions (~5 min, every participant is expected to ask one ‘non-trivial’ question)

  36. Schedule • Sep 27 - Fabric Management (2) Resource Management (2) • Oct 11 - Information Services (2) Security (2)

  37. A taste of VM Technology • VMware • Virtual PC (Microsoft) • User Mode Linux (UML)

  38. Evaluation • VMware (√) • Pro: performance, networking, multi-platform • Con: memory consumption • Virtual PC • Pro: memory consumption • Con: networking, performance, uni-platform • User Mode Linux • Open source

  39. VMware

  40. Virtual PC

  41. User Mode Linux

  42. http://www.liacs.nl/~hli/professional/courses/grid2004/lucgrid.htmhttp://www.liacs.nl/~hli/professional/courses/grid2004/lucgrid.htm

More Related