1 / 21

Grid Computing

Josva Kleist Danish Center for Grid Computing www.dcgc.dk. Grid Computing. The ATLAS experiment. Agenda. E-science Grid Computing An example Grid – NorduGrid ARC Demo Internals of NorduGrid Future. E-science.

bryce
Download Presentation

Grid Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Josva Kleist Danish Center for Grid Computing www.dcgc.dk Grid Computing Grid Computing - AAU 14/11-05

  2. The ATLAS experiment

  3. Grid Computing - AAU 14/11-05 Agenda • E-science • Grid Computing • An example Grid – NorduGrid ARC • Demo • Internals of NorduGrid • Future

  4. E-science ”Science (increasingly) done through distributed global collaborations enabled by the Internet, using very large data collections, tera-scale computing resources and high performance visualisation.”

  5. E-science the old fashioned way +

  6. Grid Computing - AAU 14/11-05 The grand vision A huge virtual distributed computer.

  7. Grid Computing - AAU 14/11-05 Definition 1 “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” The Grid – a blueprint for a new computing infrastructure, 1998

  8. Grid Computing - AAU 14/11-05 Definition 2 “The real and specific problem that underlies the Grid concept is coordinated ressource sharing and problem solving in dynamic, multiinstitutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem solving and resource-brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization.” The anatomy of the Grid, 2000

  9. Grid Computing - AAU 14/11-05 Keysentences coordinates resources that are not subject to centralized control … … using standard, open, general-purpose protocols and interfaces … … to deliver nontrivial qualities of service.

  10. Challenges • Make hardware owned by different organizations available to non-members of that organization. • In such a way that normal operation of the equipment can continue. • In such a way that the organization still can control who gets access. • In such a way that we can control who gets access to specific pieces of data. • In such a way that operations can be performed anonymously. • And still charge for the use of hard- and software.

  11. Grid Computing - AAU 14/11-05 Challenges Resource allocation and scheduling Authentication and authorization Protection Control Accounting

  12. Grid Computing - AAU 14/11-05 Globus An open source software toolkit used for building grids. Includes software services and libraries for resource monitoring, discovery, and management, plus security and file management. Web: www.globus.org

  13. Grid Computing - AAU 14/11-05 The globus model

  14. Grid Computing - AAU 14/11-05 NorduGrid • NorduGrid is a collaboration between a number of universities mostly located in the Nordic contries. • NorduGrid Advanced Resource Connector is: • A Globus-based Grid middleware solution of choice in Scandinavia and Finland • NorduGrid is a production Grid • Approximately 5000 CPUs • Approximately 75 TB of storage Web: www.nordugrid.org

  15. Grid Computing - AAU 14/11-05 ARC Components

  16. Workflow RSL RSL RC SE Gatekeeper GridFTP SE Front-end Cluster UI+RB Grid Manager MDS Source NorduGrid.org

  17. Grid Computing - AAU 14/11-05 Front-end

  18. Grid Computing - AAU 14/11-05 The user-interface ngsub to submit a task ngstat to obtain the status of jobs and clusters ngcat to display the stdout or stderr of a running job ngget to retrieve the result from a finished job ngkill to cancel a job request ngclean to delete a job from a remote cluster ngrenew to renew user’s proxy ngsync to synchronize the local job info with the MDS ngcopy to transfer files to, from and between clusters ngremove to remove files

  19. Grid Computing - AAU 14/11-05 Broker The user must be authorized to use the cluster and the queue The cluster’s and queue’s characteristics must match the requirements specified in the xRSL string (max CPU time, required free disk space, installed software etc) If the job requires a file that is registered in a Replica Catalog, the brokering gives priority to clusters where a copy of the file is already present From all queues that fulfills the criteria one is chosen randomly, with a weight proportional to the number of free CPUs available for the user in each queue If there are no available CPUs in any of the queues, the job is submitted to the queue with the lowest number of queued job per processor

  20. Grid Computing - AAU 14/11-05 Demo

  21. Grid Computing - AAU 14/11-05 To-do • Better resource brokering. • Accounting. • Scheduling. • Security. • Monitoring.

More Related