Introduction to the Grid

1. Introduction to the Grid John Kewley Grid Technology Group e-Science Centre

2. Outline • What is the Grid? • What is e-Science? • What is the NW-GRID?

3. Motivation • Many scientific communities are collaborating to share data and models. • Experimental science uses ever more sophisticated sensors to produce increasingly large amounts of experimental data This global need for access to significant amounts of data demands a more federated approach to data storage and analysis.

4. Motivation (2) Many large challenges require such community effort: • Physicists / astronomers all over the world share resources for storage and analyses of petabytes of data • Climate scientists visualize, annotate, & analyze terabyte simulation datasets • An emergency response team combines current sensor data, weather model and population data

5. Clusters and Distributed Computing • Cluster • Tightly coupled • Homogeneous • Cooperative working • Distributed Computing • Loosely coupled • Heterogeneous • Single Administration • Grid Computing • Large scale • Cross-organizational • Geographical distribution • Distributed Management Source: Hiro Kishimoto GGF17 Keynote May 2006

6. What is a Grid? A distributed (potentially geographically) collection of computational, data and/or sensor resources with are not centrally managed. International instruments,.. International grid (EGEE) Wider collaboration greater resources Regional grids ( NorduGrid) National datacentres, HPC, instruments National grids Institutes’ data, Condor pools, clusters Campus grids Desktop

7. Mobile Access G R I D M I D D L E W A R E Supercomputer, PC-Cluster Workstation Data-storage, Sensors, Experiments Visualising Internet, networks The Grid Metaphor

8. e-Science Collaborative research that is made possible by the sharing across the Internet of resources (data, instruments, computation, people’s expertise...) • Crosses organisational boundaries • Often very compute intensive • Often very data intensive • Sometimes large-scale collaboration Early examples were in science: “e-science” Relevance of “e-science technologies” to new user communities (social science, arts, humanities…) led to the term “e-research”

9. e-Research Goal: to enable better research in all disciplines Method: Develop collaboration supported by advanced distributed computation and storage • to generate, curate and analyse rich data resources • From experiments, observations and simulations • Quality management, preservation and reliable evidence • to develop and explore models and simulations • Computation and data at all scales • Trustworthy, economic, timely and relevant results • to enable dynamic distributed collaboration • Facilitating collaboration with information and resource sharing • Security, trust, reliability and accountability

10. Digital Curation Centre Centres of Excellence e-Science Centres in the UK Lancaster York Other Centres e-Science Centres NeSC AccessGrid Support Centre Newcastle Belfast Leicester National Centre for Text Mining Manchester National Centre for e-Social Science National Institute for Environmental e-Science Daresbury Cambridge Birmingham Oxford National Grid Service Cardiff RAL Bristol London Southampton Open Middleware Infrastructure Institute Reading London UK e-Science community

11. Data sharing and integration • Life sciences, sharing standard data-sets, combining collaborative data-sets • Medical informatics, integrating hospital information systems for better care and better science • Sciences, high-energy physics Source: Hiro Kishimoto GGF17 Keynote May 2006 Simulation-based science and engineering • Earthquake simulation Capability computing • Life sciences, molecular modeling, tomography • Engineering, materials science • Sciences, astronomy, physics High-throughput, capacity computing for • Life sciences: BLAST, CHARMM, drug screening • Engineering: aircraft design, materials, biomedical • Sciences: high-energy physics, economic modeling Grids in Use

12. NW-GRID NWDA funded collaboration between • University of Manchester • University of Liverpool • Lancaster University • CCLRC Daresbury Laboratory Aims: • Establish, for the region, a world-class activity in the deployment and exploitation of Grid middleware • Realise the capabilities of the Grid in leading edge academic, industrial and business computing applications

13. NW-GRID - Hardware Compute clusters from Streamline Computing Twin Sun x4200 head node • Dual-processor single-core Opteron 2.6GHz (64bit) • 16GB memory Sun x4100 worker nodes • Dual-processor dual-core AMD Opteron 2.4GHz (64bit) • 2 or 4GB memory per core • 2x 73GB disks per node • Gbit/s ethernet between nodes Panasas Storage Cluster • Around 3–10TB per cluster (Manchester have software client only)

14. NW-GRID Core Systems Manchester University man2.nw-grid.ac.uk – 27 nodes The University of Liverpool lv1.nw-grid.ac.uk - 44 nodes University of Lancaster lancs1.nw-grid.ac.uk - 192 nodes (96 dedicated to local users) CCLRC Daresbury Laboratory dl1.nw-grid.ac.uk - 96 nodes (some locally funded)

15. NW-GRID Software SUSE 9.1 Linux Sun Grid Engine (SGE) for job submission SCore for high performance MPI over Gigabit Ethernet DDT debugger plus Application Software such as ...

16. Connectivity

17. NW-GRID future • Daresbury • BlueGene L – 1000 processors, for software development, available soon. • Liverpool • POL Cluster – 192x Xeon (32 bit), likely to become part of NW-GRID in April. • Preston • 8-node SGI Altix 3700. Discussions underway with Dept. of Theoretical Astrophysics, U. Central Lancashire.

18. Links What is the Grid? http://gridcafe.web.cern.ch/ What is e-Science? http://www.e-science.cclrc.ac.uk/ http://www.nesc.ac.uk/ What is the NW-GRID? http://www.nw-grid.ac.uk/

19. Acknowledgements Slides are from various sources including Rob Allan, NeSC, EGEE, ...