1 / 55

Introduction to Grid Computing and Applications in Computational Sciences

Introduction to Grid Computing and Applications in Computational Sciences. Barry Wilkinson Department of Mathematics and Computer Science Western Carolina University. WSSU September 13, 2004. Talk Outline. What is Grid Computing? Applications Evolution of Grid Computing

tyme
Download Presentation

Introduction to Grid Computing and Applications in Computational Sciences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Grid Computing and Applications in Computational Sciences Barry Wilkinson Department of Mathematics and Computer Science Western Carolina University WSSU September 13, 2004

  2. Talk Outline • What is Grid Computing? • Applications • Evolution of Grid Computing • Grid Computing Course

  3. Grid Computing • Using usually geographically distributed and interconnected computers together for high performance computing and/or for resource sharing. Notice “usually”, “and/or” - many definitions of grid computing and applications.

  4. The interconnection - now “usually” made through the Internet to multiple administrative domains. Resource sharing - can involve geographically distributed resources in addition to computers such as software, experimental equipment etc.

  5. Some think that grid computing is just cluster computing in the “large”

  6. Scalable Computing 2100 2100 2100 2100 2100 2100 2100 2100 2100 PERFORMANCE + Q o S Administrative Barriers • Individual • Group • Department • Campus • State • National • Globe • Inter Planet • Universe Inter Planet Grid Personal Device Local Cluster Enterprise Cluster/Grid Global Grid SMPs or SuperComputers Figure due to Rajkumar Buyya, University of Melbourne, Australia, www.gridbus.org

  7. But grid computing is more than this. It offer the potential of virtual organizations • groups of people both geographically and organizationally distributed working together on a problems sharing computers AND other resources such as databases and experimental equipment.

  8. The grid virtualizes heterogeneous geographically disperse resources From "Introduction to Grid Computing with Globus," IBM Redbooks

  9. Distributed Collaborative Experiment Figure from M. Faramawi and B. Ramamurthy, SUNY- Buffalo

  10. Australia Nimrod-G Gridbus GridSim Virtual Lab DISCWorld GrangeNet. ..etc Europe UK eScience EU Data Grid Cactus XtremeWeb ..etc. India I-Grid Japan Ninf DataFarm Korea... N*Grid Singapore NGP USA AppLeS Globus Legion Sun Grid Engine NASA IPG Condor-G Jxta NetSolve AccessGrid and many more... Cycle Stealing & .com Initiatives Distributed.net SETI@Home, …. Entropia, UD, SCS,…. Public Forums Global Grid Forum Australian Grid Forum IEEE TFCC CCGrid conference P2P conference Some Grid Projects & Initiatives Figures due to Rajkumar Buyya, University of Melbourne, Australia, www.gridbus.org http://www.gridcomputing.com

  11. Example Grid Networks • Numerous very high performance computing projects developed in late 1990’s and 2000’s. • Examples: USA TeraGrid, UK e-Science Grid, and others

  12. TeraGrid

  13. TeraGrid

  14. UK e-Science Grid

  15. EU grid

  16. Computational Grid Applications • Biomedical research • Industrial research • Engineering research • Studies in Physics and Chemistry

  17. Some “Computational” Grid Projects • Large Hadron Collider experimental facility for complex particle experiments at CERN (European Center for Nuclear Research, near Geneva Switzerland). • DOE Particle Physics Data grid • DOE Science grid • AstroGrid Project • Comb-e-Chem project

  18. CERN grid

  19. Key aspects of these grids • State-of-the-art interconnection networks. • Sharing resources. • Community of scientists.

  20. Shared Resources Can be much more than just computers: • Storage • Sensors for experiments at particular sites in the grid • Application Software • Databases, ...

  21. Resource sharing and collaborative computing • Grid computing is about collaborating and resource sharing as much as it is about high performance computing. • Many projects

  22. Key aspects • Using distributed computers and resources collectively. • Often crossing organizational boundaries • Fueled by the Internet providing communication network.

  23. Evolution of grid computing • Started as a form of distributed computing. • Previous distributed computing systems: • 1980’s - Remote Procedure calls (RPC) client -server model with a service registry. • 1990’s - Distributed objects systems: • CORBA (Common Request Broker Architecture) • Java RMI (Remote Method Invocation)

  24. Internet-Based Grid computing • Grid Computing is now based upon Internet. • Enables using existing Internet protocols,security mechanisms, etc. • Uses a form of web services.

  25. Applications • Originally e-Science applications • Computational intensive, not necessarily one big problem but a problem that has to be solved repeatedly. • Data intensive. • Experimental collaborative projects • Now also e-Business applications to improve business models and practices.

  26. Background • Emergence and immense success of the Internet and the world-wide web, with agreed upon Internet standards for communication and access. • Continual improvement on computer and network technology and speeds.

  27. Need to harness computers • Original driving force behind Internet same as grid computing! • the need for high performance computing by connecting computers at distributed sites.

  28. Economic Development • Cohen report: September 2003, • projected impact of grid computing on NC’s economy - could lead to 240,000 new jobs and $10 billion in economic growth in North Carolina by 2010.

  29. Grid Computing CourseFall 2004Barry Wilkinson Western Carolina UniversityandClayton FernerUNC-Wilmington

  30. Originates from WCU on NCREN network and broadcast to students and faculty at 8 participating institutions: • UNC-Wilmington • NC State University • UNC-Asheville • UNC-Greensboro • Appalachian State University • NC Central University • Cape Fear Community College • Elon University

  31. Level Listed as an undergraduate course but can be taken for graduate credit. Graduate students expected to do more demanding work. Most students are undergraduate, but there are a few graduate students at NCSU and UNC-W.

  32. Prerequisites • Preferably programming skills in Java on a Linux system. • Some later work involves C/C++ programming.

  33. Topics • Review of Internet technologies • Introduction to grid computing • Web services • Grid services • Security, Public Key Infrastructure • Open Grid Services Architecture (OGSA) • Globus 3.2 • Condor-G • MPI and grid-enabled MPI • UNC-W workflow editor and other GUI tools • Grid computing applications

  34. Assessment • 6 “simple” programming tasks • Creating a web service • Creating a grid service • Submitting a Globus job • Submitting a CondorG job • MPI-G2 program • Using UNC-W GUI workflow editor • Programming Project • Class tests (1 or 2) • Final test • Small print: Subject to change. The instructor reserves the right to change the assignments to make it easier or harder.

  35. Weeks 1 - 3 Grid computing Virtual organizations, computational grid projects, grid computing networks, TeraGrid, grid projects in the US and around the world, grid challenges Internet Technologies IP addresses, HTTP, URL, HTTP, XML, Telnet, FTP, SSL Web Services I. Service-Oriented Architecture (SOA), service registry, XML documents, XML schema, namespaces, SOAP, XML/SOAP examples, AxisWeb Services II. WSDL, portType, message definition, WSDL to/from code Assignment 1"Simple" Web service Java programming assignment. Tomcat environment, axis, JWS facility

  36. Weeks 3 - 4 Grid Service Concepts, differences to Web services, stateful/stateless/transient/non-transient, Open Grid Services Architecture (OGSA), OGSI, grid service factory, Web Services Resource Framework (WSRF) Assignment 2"Simple" grid service Java programming assignment. Globus 3.2 environment.Tools: ant

  37. Weeks 4 - 6 SecuritySecure connection, authorization requirements, symmetric and asymmetric (public/private) key cyptography, non-repudiation, digital signatures, certificates, certificate authorities, X509 certificate Globus: Part 1 Basic structure (version 3.2), grid service container, service browser, Globus Resource Allocation Manager (GRAM), job submission with managed-job-globusrun, Grid Security Infrastructure (GSI), Globus certificates, simpleCA, proxies, creating a proxy Globus: Part IIResource management, Master Managed Job Factory Service (MMJFS), more on managed-job-globusrun. Resource Specification Language (RSL and RSL-2), syntax and examples in RSL and RSl-2 Assignment 3 Submitting a Job to the Grid, GT3 mangaged-job-globusrun, job specified in RSL-2 (XML file)

  38. Weeks 6 - 7 Globus: Part IIIInformation Directory Services, LDAP, resource discovery Schedulers and Condor, submit description file, resource brokers DAGMan, Checkpointing, ClassAd, Condor-G, other systems Assignment 4 Submitting a Condor-G Job

  39. Weeks 7 - 8 High performance Grand challenge problems, parallel computing (HPF) computing, potential speed-up, types of parallel computers, shared memory multiprocessors, programming, message-passing multicomputers Parallel ProgrammingTechniques suitable for a Grid, embarrassingly parallel computations, Monte Carlo, parameter studies, sample "big" problems, gravitational N-body problem Cluster ComputingBasic message passing techniques, History, Beowulf clusters, system software, programming models (MPMD, SPMD), synchronous message passing, asynchronous message passing, message tags, collective routines

  40. Weeks 8 - 9 MPI Process creation, communicators, unsafe message passing, point-to-point message-passing, blocking, non-blocking, communication modes, collective communication, running an MPI program on a cluster Grid-enabled MPIMPI-G2 internals, mpirun command, RSL script Assignment 5Running a simple MPI-G2 program

  41. Weeks 10 to 15 Grid portals UNC-W GUI Scientific and business applications Guest Speaker: Professor Dan Reed, University of North Carolina, Chapel, NC State University, and Duke University.

  42. Course Text • There is no assigned course textbook • Materials and links are provided on the home page.

  43. Course Home Page http://www.cs.wcu.edu/~abw/CS493F04 for announcements, slides, assignments, reading materials, tests dates, etc.

  44. Acknowledgements This course is a team effort of: Mountain Area Grid Innovation Collaborative (MAGIC) Faculty: Barry Wilkinson and Mark Holliday Students (Wizards): Jeffrey House and Sam Daoud http://www.cs.wcu.edu/~abw/MAGIC and: University of North Carolina at Wilmington

More Related