1 / 42

State-Wide Collaborative Grid Computing Course

State-Wide Collaborative Grid Computing Course. Barry Wilkinson Department of Computer Science University of North Carolina at Charlotte and Mark Holliday Department of Mathematics and Computer Science Western Carolina University. Participating Sites.

wiley
Download Presentation

State-Wide Collaborative Grid Computing Course

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. State-Wide Collaborative Grid Computing Course Barry Wilkinson Department of Computer Science University of North Carolina at Charlotte and Mark Holliday Department of Mathematics and Computer Science Western Carolina University Participating Sites Teaching and Learning with Technology Conference March 30th – April 1st, 2005, Raleigh, NC.

  2. Talk Outline • Brief introduction to Grid Computing. • Software Tools. • Grid Computing Course.

  3. Grid Computing Using geographically distributed interconnected computers for high performance computing and/or for resource sharing. The grid virtualizes heterogeneous geographically disperse resources From "Introduction to Grid Computing with Globus," IBM Redbooks

  4. Virtual Organizations Grid computing offers potential of virtual organizations: • groups of people, both geographically and organizationally distributed, working together on a problem, sharing computers AND other resources such as databases and experimental equipment. • Crosses multiple administrative domains.

  5. Shared Resources Can share much more than just computers: • Storage • Sensors for experiments at particular sites • Application Software • Databases • Network capacity, …

  6. Interconnections and Protocols Focus now on: • using standard Internet protocols and technology, i.e. HTTP, SOAP, web services, etc., • Grid computing standards around web services.

  7. Applications • Originally e-Science applications • Computational intensive • Not necessarily one big problem but a problem that has to be solved repeatedly with different parameters. • Data intensive. • Experimental collaborative projects • Now also e-Business applications to improve business models and practices.

  8. History • Began in mid 1990’s with experiments using computers at geographically dispersed sites. • Seminal experiment – “I-way” experiment at 1995 Supercomputing conference (SC’95), using 17 sites across the US running: • 60+ applications. • Existing networks (10 networks).

  9. Grid Networks and Projects • Numerous very high performance computing projects developed in late 1990’s and 2000’s. • Examples: USA TeraGrid, UK e-Science Grid, and many others.

  10. TeraGrid

  11. UK e-Science Grid

  12. Software - Globus Project • Open source software toolkit developed for grid computing. • Roots in I-way experiment. • Work started in 1996. • Four versions developed to present time. • Reference implementations of grid computing standards. • Defacto standard for grid computing.

  13. Globus Key Components • GSI (Grid Security Infrastructure) • Grid security. • MDS (Monitoring and discovery Service) • Interface to system and service information. • GRAM (Grid Resource Allocation Manager) • Remote job submission and control. • GridFTP • Secure data transfer.

  14. 2. discover resource MDS 3. submit job GRAM 4. transfer data GridFTP secure environment GSI From “Globus Toolkit 4 Tutorial,” MCNC Jan-Feb, 2005, Pawel Plaszczak and Bogdan Lobodzinski, Gridwise Technologies.

  15. Globus Toolkit: Recent History • GT2 (2.4 released in 2002) • GRAM, MDS, GridFTP, GSI. • GT3 (3.2 released mid-2004): redesign • OGSA (Open Grid Service Architecture) - OGSI (Open Grid Services Infrastructure) based. • Introduced “Grid services” as an extension of web services. • OGSI now abandoned. • GT4 (release for April 2005): redesign • WSRF (Web service Resource Framework) based. • Grid standards merged with Web services.

  16. Supercomputing 2003 Demonstration • We used Globus version 2.4 in a Supercomputing 2003 demo organized by the University of Melbourne. • 21 countries involved, numerous sites.

  17. Version 3 A re-implementation based upon the Open Grid Service Architecture (OGSA) standard. • We used version 3.2 for the Fall 2004 course. • Underlying implementation of version 3.x used OGSI Open Grid Service Infrastructure), which was not embraced by the community.

  18. Version 4 • Currently under development to be released early 2005. • OGSA kept but OGSI abandoned in favor of new implementation standards based around web services. (Version 3 used “extended” web services)

  19. Web Services-Based Grid Computing • Grid Computing is now strongly based upon web services. • Large number of newly proposed grid computing standards: • WS-Resource Framework • WS-Addressing • etc., etc. …. .

  20. Grid Computing Course (Fall 2004) • Originated from WCU on NCREN network. Broadcast to: • UNC-Wilmington • NC State University • UNC-Asheville • UNC-Greensboro • Appalachian State University • NC Central University • Cape Fear Community College • Elon University • UNC-Charlotte* • Instructors: • Barry Wilkinson and Clayton Ferner (UNC-Wilmington) • Several faculty helped at various sites • 43 students (*canceled)

  21. Participating Sites

  22. Level and Prerequisites • Listed as an undergraduate course but can be taken for graduate credit. (Graduate students to do more work.) • Preferably have programming skills in Java on a Linux system.

  23. Topics • Review of Internet technologies • Introduction to grid computing • Web services • Grid services (OGSA) • Security, Public Key Infrastructure • Globus 3.2 • Condor-G • MPI and grid-enabled MPI • UNC-W workflow editor and other GUI tools • Grid computing applications 5 grid computing assignments. Software from NMI package.

  24. Weeks 1 - 3 Grid computing Virtual organizations, grid projects in the US and around the world, grid challenges. Internet IP addresses, HTTP, URL, HTTP, Technologies XML, Telnet, FTP, SSL. Web Services Service-Oriented Architecture, service registry, XML documents, XML schema, namespaces, SOAP, XML/SOAP examples, WSDL, WSDL to/from code. Assignment 1“Simple” Web service Java programming assignment. Tomcat environment, axis, JWS facility.

  25. Weeks 3 - 4 Grid Service Concepts, differences to Web services, stateful/ stateless/transient/non-transient, Open Grid Services Architecture (OGSA), OGSI, grid service factory, Web Services Resource Framework (WSRF) Assignment 2"Simple" grid service Java programming assignment. Globus 3.2 environment.Tools: ant.

  26. Weeks 4 - 6 SecuritySecure connection, authorization requirements, symmetric and asymmetric key cyptography, non-repudiation, digital signatures, X 509 certificates, certificate authorities. Globus: Basic structure, grid service container, service browser, Resource Allocation Manager, job submission with managed-job-globusrun, Grid Security Infrastructure, Globus certificates, simpleCA, proxies, creating a proxy, Assignment 3 Submitting a Job to the Grid, GT3 mangaged-job-globusrun, job specified in RSL-2 (XML file).

  27. Weeks 6 - 7 Resource Master Managed Job Factory Management Service, Resource Specification Language (RSL and RSL-2).Information Directory Services, LDAP, resource discovery. Schedulers and Condor, submit description resource brokers file, DAGMan, checkpointing, ClassAd, Condor-G. Assignment 4 Submitting a Condor-G Job

  28. Weeks 7 - 8 High performance Grand challenge problems, computing (HPF) parallel computing, potential speed-up, types of parallel computers. Parallel Techniques suitable for a Grid, Programming embarrassingly parallel computations, Monte Carlo, parameter studies, sample "big" problems, gravitational N-body problem Cluster ComputingBasic message passing techniques, programming models, synchronous message passing, asynchronous message passing, message tags, collective routines.

  29. Weeks 8 - 9 MPI Process creation, communicators, unsafe message passing, MPI routines, running MPI program on a cluster. Grid-enabled MPIMPI-G2 internals, mpirun command, RSL script Assignment 5Running a simple MPI-G2 program (not done Fall 2004).

  30. Weeks 10 to 11 Grid Portal Design • “A web-based application server enhanced with the necessary software to communicate to grid services and resources” • “Provides application scientist a customized view of software and hardware resources from a web browser”

  31. OGCEGrid

  32. Weeks 11 to 12 Workflow Technique • Functional decomposition - dividing problem into separate functions that take results from other functional units and pass on results onto functional units. • Interconnection patterns depends upon the problem. • Workflow - describes the flow of information between the units.

  33. Workflow Example - Climate Modeling

  34. GridNexus Workflow Editor • Developed by UNC-Wilmington. • Introduced in course. • Assignment set. www.gridnexus.org

  35. GridNexus Workflowusing Grid Services

  36. Weeks 14 to 15 Guest Speakers • Professor Daniel A. Reed, Chancellor's Eminent Professor, Vice Chancellor for IT and CIO, UNC-Chapel Hill, Director of Institute for Renaissance Computing, University of North Carolina at Chapel Hill, Duke University, and NC State University: • “Grid computing: 21st Century Challenges.” • Dr. Wolfgang Gentzsch, Managing Director, MCNC Grid Computing and Networking Services: • “Grid Computing in the Industry” • Chuck Kesler, Director, Grid Deployment and Data Center Services, MCNC: • “Security Policy, Legal, and Regulatory Challenges in Grid Computing Environments” • Professor Ian Foster: Ian Foster, Argonne National Laboratory and University of Chicago: • “The Grid: Beyond the Hype” Taped presentation (originally given at Duke University, Sept. 14th, 2004).

  37. Course Home Page http://www.cs.wcu.edu/~abw/CS493F04 WebCT also used for quizzes.

  38. Fall 2005 Grid Computing Course • To originate from UNC-Charlotte. • Again in collaboration with UNC-Wilmington. • Can be for undergraduate or graduate credit.

  39. Acknowledgements Partial support for this work was provided by the National Science Foundation and University of North Carolina, Office of the President. • National Science Foundation, “Introducing Grid Computing into the Undergraduate Curricula,” ref. DUE 0410667, PI: A. B. Wilkinson, $100,000, 2004-2006. • University of North Carolina Office of President, “A Consortium to Promote Computational Science and High Performance Computing,” PI: B. Kurtz (Appalachian State University) co-PI B. Wilkinson and others at various universities, total $650,000, 2004-2006. • University of North Carolina Office of President, “Fostering Undergraduate Research Partnerships through a Graphical User Environment for the North Carolina Computing Grid,” PI: R. Vetter (UNC-Wilmington), co-PI B. Wilkinson and others at various universities, total $557,634, 2004-2006.

  40. Papers Since Fall 2004 • B. Wilkinson, M. Holliday, and C. Ferner, “Experiences in Teaching a Geographically Distributed Undergraduate Grid Computing Course,” Workshop on Collaborative and Learning Applications of Grid Technology and Grid Education, IEEE International Symposium on Cluster Computing and the Grid (CCGrid2005), Cardiff, UK, May 9 - 12, 2005, accepted. • M. A. Holliday, B. Wilkinson, J. House, S. Daoud, and C. Ferner, “A Geographically-Distributed, Assignment-Structured Undergraduate Grid Computing Course,” SIGCSE 2005 Technical Symposium on Computer Science Education, St. Louis, Missouri, February 23 - 27, 2005.

  41. Questions?

More Related