190 likes | 344 Views
Introduction to the Grid. John Kewley Grid Technology Group e-Science Centre. Outline. What is the Grid? What is e-Science? What is the NW-GRID?. Motivation. Many scientific communities are collaborating to share data and models.
E N D
Introduction to the Grid John Kewley Grid Technology Group e-Science Centre
Outline • What is the Grid? • What is e-Science? • What is the NW-GRID?
Motivation • Many scientific communities are collaborating to share data and models. • Experimental science uses ever more sophisticated sensors to produce increasingly large amounts of experimental data This global need for access to significant amounts of data demands a more federated approach to data storage and analysis.
Motivation (2) Many large challenges require such community effort: • Physicists / astronomers all over the world share resources for storage and analyses of petabytes of data • Climate scientists visualize, annotate, & analyze terabyte simulation datasets • An emergency response team combines current sensor data, weather model and population data
Clusters and Distributed Computing • Cluster • Tightly coupled • Homogeneous • Cooperative working • Distributed Computing • Loosely coupled • Heterogeneous • Single Administration • Grid Computing • Large scale • Cross-organizational • Geographical distribution • Distributed Management Source: Hiro Kishimoto GGF17 Keynote May 2006
What is a Grid? A distributed (potentially geographically) collection of computational, data and/or sensor resources with are not centrally managed. International instruments,.. International grid (EGEE) Wider collaboration greater resources Regional grids ( NorduGrid) National datacentres, HPC, instruments National grids Institutes’ data, Condor pools, clusters Campus grids Desktop
Mobile Access G R I D M I D D L E W A R E Supercomputer, PC-Cluster Workstation Data-storage, Sensors, Experiments Visualising Internet, networks The Grid Metaphor
e-Science Collaborative research that is made possible by the sharing across the Internet of resources (data, instruments, computation, people’s expertise...) • Crosses organisational boundaries • Often very compute intensive • Often very data intensive • Sometimes large-scale collaboration Early examples were in science: “e-science” Relevance of “e-science technologies” to new user communities (social science, arts, humanities…) led to the term “e-research”
e-Research Goal: to enable better research in all disciplines Method: Develop collaboration supported by advanced distributed computation and storage • to generate, curate and analyse rich data resources • From experiments, observations and simulations • Quality management, preservation and reliable evidence • to develop and explore models and simulations • Computation and data at all scales • Trustworthy, economic, timely and relevant results • to enable dynamic distributed collaboration • Facilitating collaboration with information and resource sharing • Security, trust, reliability and accountability
Digital Curation Centre Centres of Excellence e-Science Centres in the UK Lancaster York Other Centres e-Science Centres NeSC AccessGrid Support Centre Newcastle Belfast Leicester National Centre for Text Mining Manchester National Centre for e-Social Science National Institute for Environmental e-Science Daresbury Cambridge Birmingham Oxford National Grid Service Cardiff RAL Bristol London Southampton Open Middleware Infrastructure Institute Reading London UK e-Science community
Data sharing and integration • Life sciences, sharing standard data-sets, combining collaborative data-sets • Medical informatics, integrating hospital information systems for better care and better science • Sciences, high-energy physics Source: Hiro Kishimoto GGF17 Keynote May 2006 Simulation-based science and engineering • Earthquake simulation Capability computing • Life sciences, molecular modeling, tomography • Engineering, materials science • Sciences, astronomy, physics High-throughput, capacity computing for • Life sciences: BLAST, CHARMM, drug screening • Engineering: aircraft design, materials, biomedical • Sciences: high-energy physics, economic modeling Grids in Use
NW-GRID NWDA funded collaboration between • University of Manchester • University of Liverpool • Lancaster University • CCLRC Daresbury Laboratory Aims: • Establish, for the region, a world-class activity in the deployment and exploitation of Grid middleware • Realise the capabilities of the Grid in leading edge academic, industrial and business computing applications
NW-GRID - Hardware Compute clusters from Streamline Computing Twin Sun x4200 head node • Dual-processor single-core Opteron 2.6GHz (64bit) • 16GB memory Sun x4100 worker nodes • Dual-processor dual-core AMD Opteron 2.4GHz (64bit) • 2 or 4GB memory per core • 2x 73GB disks per node • Gbit/s ethernet between nodes Panasas Storage Cluster • Around 3–10TB per cluster (Manchester have software client only)
NW-GRID Core Systems Manchester University man2.nw-grid.ac.uk – 27 nodes The University of Liverpool lv1.nw-grid.ac.uk - 44 nodes University of Lancaster lancs1.nw-grid.ac.uk - 192 nodes (96 dedicated to local users) CCLRC Daresbury Laboratory dl1.nw-grid.ac.uk - 96 nodes (some locally funded)
NW-GRID Software SUSE 9.1 Linux Sun Grid Engine (SGE) for job submission SCore for high performance MPI over Gigabit Ethernet DDT debugger plus Application Software such as ...
NW-GRID future • Daresbury • BlueGene L – 1000 processors, for software development, available soon. • Liverpool • POL Cluster – 192x Xeon (32 bit), likely to become part of NW-GRID in April. • Preston • 8-node SGI Altix 3700. Discussions underway with Dept. of Theoretical Astrophysics, U. Central Lancashire.
Links What is the Grid? http://gridcafe.web.cern.ch/ What is e-Science? http://www.e-science.cclrc.ac.uk/ http://www.nesc.ac.uk/ What is the NW-GRID? http://www.nw-grid.ac.uk/
Acknowledgements Slides are from various sources including Rob Allan, NeSC, EGEE, ...