370 likes | 434 Views
Edinburgh - at the Frontiers of e-Science. Richard Kenway. discovery science. e-science = searching for the unknown. in vast amounts of data. electronic ‘needle in a haystack’. to find the Higgs boson and explain where mass comes from. and. … are not enough. you need to build a Grid.
E N D
Edinburgh - at the Frontiers of e-Science Richard Kenway
discovery science e-science = searching for the unknown in vast amounts of data
electronic ‘needle in a haystack’ • to find the Higgs boson • and explain where mass comes from and … are not enough • you need to build a Grid
LHC computing challenge assumes PC = ~ 25 SpecInt95 ~PByte/sec Online System ~100 MByte/sec Offline Farm~20,000 PCs • one bunch crossing per 25 ns • 100 triggers per second • each event is ~1 MByte ~100 MByte/sec Tier 0 CERN Computer Centre >20,000 PCs ~ Gbit/sec or Air Freight HPSS Tier 1 RAL Regional Centre US Regional Centre Italian Regional Centre French Regional Centre HPSS HPSS HPSS HPSS Tier 2 Tier2 Centre ~1000 PCs Tier2 Centre ~1000 PCs Tier2 Centre ~1000 PCs ScotGRID++ ~1000 PCs ~Gbit/sec Tier 3 Institute ~200 PCs physicists work on analysis “channels” each institute has ~10 physicists working on one or more channels data for these channels is cached by the institute server Institute Institute Institute Physics data cache 100 - 1000 Mbit/sec Tier 4 Workstations
the web on steroids • 1989: Tim Berners-Lee invented the web • so physicists around the world could share documents • 1999: Grids add to the web • computing power • data management • big instruments • (eventually) sensors
software computers sensor nets instruments colleagues data archives a new global infrastructure • the Grid is an emergent infrastructure to deliver dependable, pervasive and uniform access to globally distributed, dynamic and heterogeneous resources • problems of scalability, interoperability, fault tolerance, resource management and security • information on demand - like power from a socket
why now? 3.5 million users 22 teraflops • for 50 years, we have been riding the crest of a IT wave • building vast untapped global resources • hundreds of millions of (mostly) idle PCs and • big science is facing a data tsunami
MIPS/chip 1,000,000 100,000 P12 10,000 1,000 P8 P7 (Merced) Pentium Pro 100 Pentium* 486* 10 386* 1 286* Year 0 2005 2010 1985 1995 1990 2000 MIPS - Millions of instructions per second *Pentium, 286, 386 and 486 are registered trademarks of Intel Corp. increase in MIPS per chip microprocessor speeds double every 18 months (Moore’s Law)
(million) actual and projected 180 150 120 85 56.2 36.7 26.1 16.7 8.2 Jul-95 Jul-96 Jul-97 Jul-98 Jul-99 Jul-00 Jul-01 Jul-02 Jul-03 Source: ITU “Challenges to the Network: Internet for Development, 1999” Internet Software Consortium (www.isc.org), RIPE (www.ripe.net) internet hosts network capacity doubles every 9 months
1,200 fixed-line telephones mobile phones 1,000 estimated Internet users 800 600 400 200 0 1999 2002 2000 2003 1995 1997 1996 1998 2001 note: columns show actual and projected users at end of year source: ITU fixed lines, mobile phones & internet users millions
Quality of Service on the internet • aim to distinguish types of traffic • high priority fast lanes • low priority slow lanes • hard to configure • intersim simulation tool • detailed model of network • understand and validate configurations EPCC + Cisco Systems
wingmodels • lift capabilities • drag capabilities • responsiveness stabilizermodels airframe models • deflection capabilities • responsiveness crew capabilities - accuracy - perception - stamina - reaction times - SOP’s enginemodels • braking performance • steering capabilities • traction • dampening capabilities humanmodels • thrust performance • reverse thrust performance • responsiveness • fuel consumption landinggearmodels whole-system simulations NASA Information Power Grid: coupling all sub-system simulations
in-flight data global network eg SITA ground station airline DS&S Engine Health Center internet, e-mail, pager data centre maintenance centre global in-flight engine diagnostics Distributed Aircraft Maintenance Environment: Universities of Leeds, Oxford, Sheffield &York
stabilizer models GRC engine models 44,000 wing runs 50,000 engine runs wing models airframe models 66,000 stabilizer runs ARC LaRC Virtual National Air Space VNAS 22,000 commercial US flights a day 22,000 airframe impact runs • FAA ops data • weather data • airline schedule data • digital flight data • radar tracks • terrain data • surface data simulation drivers 48,000 human crew runs 132,000 landing/ take-off gear runs landing gear models human models National Airspace Simulation Environment NASA Information Power Grid: aircraft, flight paths, airport operations and the environment are combined to get a virtual national airspace
from genome to function • gene expression as an embryo develops EPCC MouseGrid: optical tomography image reconstruction in real time
digital radiology on the Grid • 28 petabytes/year for 2000 hospitals • must satisfy privacy laws University of Pennsylvania
emergency response teams • bring sensors, data, simulations and experts together • wildfire: predict movement of fire & direct fire-fighters • also earthquakes, peacekeeping forces, battlefields,… National Earthquake Simulation Grid Los Alamos National Laboratory: wildfire
Earth observation • ENVISAT • € 3.5 billion • 400 terabytes/year • 700 users • ground deformation prior to a volcano
data, information and knowledge • virtual data …from the grid • from a database somewhere • computed on request • measured on request • automated knowledge …from computer science • data: un-interpreted bits and bytes • information: data equipped with meaning • knowledge: information applied to solve a problem
Knowledge Grid Control Data to Knowledge Information Grid Computation/ Data Grid three layer Grid abstraction
the Grid as an evolving concept • enabler for transient ‘virtual organisations’ • anatomy: a software infrastructure that enables flexible, secure, co-ordinated resource sharing among dynamic collections of individuals, institutions and resources • Foster, Kesselman & Tuecke (2001) • evolution of and integration with web services • physiology: everything is a Grid service ie a service that conforms to a set of conventions for management and exchanging messages • Foster, Kesselman, Nick & Tuecke (2002) • Global Grid Forum: define a standard Grid architecture • big business and big science working together
UK e-Science programme ‘e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ ‘e-Science will change the dynamic of the way science is undertaken.’ John Taylor Director General of Research Councils Office of Science and Technology
UK e-Science funding DG Research Councils Grid TAG E-Science Steering Committee Director Director’s Awareness and Co-ordination Role Director’s Management Role Generic Challenges EPSRC (£15m), DTI (£15m) Academic Application Support Programme Research Councils (£74m), DTI (£5m) PPARC (£26m) BBSRC (£8m) MRC (£8m) NERC (£7m) ESRC (£3m) EPSRC (£17m) CLRC (£5m) £80m Collaborative projects Industrial Collaboration (£40m)
AccessGrid always-on video walls UK e-science centres Edinburgh Glasgow DL Newcastle Belfast Manchester Cambridge Oxford Hinxton RAL Cardiff London Soton
National e-Science Centre • Edinburgh + Glasgow Universities • Physics & Astronomy 2 • Informatics, Computing Science • EPCC • £6M EPSRC/DTI + £2M SHEFC over 3 years • e-Science Institute • visitors, workshops, co-ordination, outreach • middleware development • 50 : 50 industry : academia • ‘last-mile’ networking www.nesc.ac.uk
data, data everywhere… • globally distributed heterogeneous databases are growing very fast • science is at the frontier • commerce, healthcare, entertainment are not far behind • Scottish e-Data Information & Knowledge Transformation Centre (eDIKT) • proposal to SHEFC for a centre to develop scalable database tools • astronomy, bioinformatics, geophysics, particle physics & commerce
Scotland at the frontier… leading • UK core e-science • data integration • linked to US Globus • UK GridPP + ScotGrid • particle physics data analysis • linked to EU DataGrid • UK AstroGrid • virtual observatory • linked to EU AVO • EU enacts + GRIDSTART • supercomputer centres • EU grid projects
DARPA Scotland at the frontier… participating • EU DataGrid: particle physics, biology & medical imaging, Earth observation • US DARPA Control of Agent-Based Systems Grid: multinational military operations • UK RealityGrid: interactively couple experiments, simulations and visualisation over 100 scientists engaged in grid development by the end of 2002
from little acorns… “ It is worth noting that an essential feature of the type of theory which has been described in this note is the prediction of incomplete multiplets of scalar and vector bosons. ” Peter Higgs (1964) “ … a billion people interacting with a million e-businesses with a trillion intelligent devices interconnected ” Lou Gerstner, IBM (2000) another technological revolution is underway