300 likes | 377 Views
The Grid: The First 50 Years. Ian Foster Argonne National Laboratory University of Chicago Carl Kesselman Information Sciences Institute University of Southern California. Licklider (1960): Man-Computer Symbiosis….
E N D
The Grid: The First 50 Years Ian Foster Argonne National Laboratory University of Chicago Carl Kesselman Information Sciences Institute University of Southern California
Licklider (1960):Man-Computer Symbiosis… … is an expected development in cooperative interaction between men and electronic computers. The main aims are to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs.
Problem Solving in the 21st Century • Teams organized around common goals • Communities: “Virtual organizations” • With diverse membership & capabilities • Heterogeneity is a strength not a weakness • And geographic and political distribution • No location/organization possesses all required skills and resources • Must adapt as a function of the situation • Adjust membership, reallocate responsibilities, renegotiate resources
Context (1):Revolution in Science • Pre-Internet • Theorize &/or experiment, aloneor in small teams; publish paper • Post-Internet • Construct and mine large databases of observational or simulation data • Develop simulations & analyses • Access specialized devices remotely • Exchange information within distributed multidisciplinary teams
Context (2):Revolution in Business • Pre-Internet • Central data processing facility • Post-Internet • Enterprise computing is highly distributed, heterogeneous, inter-enterprise (B2B) • Business processes increasingly computing- & data-rich • Outsourcing becomes feasible => service providers of various sorts
The (Power) Grid:On-Demand Access to Electricity Quality, economies of scale Time
By Analogy, A Computing Grid • Decouple production and consumption • Enable on-demand access • Achieve economies of scale • Enhance consumer flexibility • Enable new devices • On a variety of scales • Department • Campus • Enterprise • Internet
Not Exactly a New Idea … • “The time-sharing computer system can unite a group of investigators …. one can conceive of such a facility as an … intellectual public utility.” • Fernando Corbato and Robert Fano, 1966 • “We will perhaps see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country.” • Len Kleinrock, 1967
Computing isn’t Really Like Electricity • I import electricity but must export data • “Computing” is not interchangeable but highly heterogeneous: data, sensors, services, … • This complicates things; but also means that the sum can be greater than the parts • Real opportunity: Construct new capabilities dynamically from distributed services • Raises fundamental questions • Achieving economies of scale • Quality of service across distributed services • Applications that exploit synergies
New OpportunitiesDemand New Technology “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations” “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder)
Taking Sharing to the Next Level • Sharing of communication • Telephones, mailing lists, collaboration tools • Sharing of data and knowledge • Web, semantic web • What about the rest of the infrastructure? • Services, computers, programs, sensors, …
Existing Technologies are Helpful,but Not Complete Solutions • Peer-to-peer technologies • Limited scope and mechanisms • Enterprise-level distributed computing • Limited cross-organizational support • Databases • Vertically integrated solutions • Web services • Not dynamic • Semantic web • Limited focus
What’s Missing is Support for … • Sharing & integration of resources, via • Discovery • Provisioning • Access (computation, data, …) • Security • Policy • Fault tolerance • Management • In dynamic, scalable, multi-organizational settings
Enter the Grid • Infrastructure (“middleware”) for establishing, managing, and evolving multi-organizational federations • Dynamic, autonomous, domain independent • On-demand, ubiquitous access to computing, data, and services • Mechanisms for creating and managing workflow within such federations • New capabilities constructed dynamically and transparently from distributed services • Service-oriented, virtualization
Building the Grid • Open source software • Globus Toolkit® , UK OGSA DAI, Condor, … • Open standards • OGSA, other GGF, IETF, W3C standards, … • Open communities • Global Grid Forum, Globus International, collaborative projects, … • Open infrastructure • UK eScience, NSF Cyberinfrastructure, StarLight, AP-Grid, …
Open Source • Encourage adoption of standards by reducing barriers to entry • Overcome new technology Catch-22 • Enable broad Grid technology ecosystem • Key to international cooperation • Key to science-commerce partnership • Jumpstart Grid industry and allow vendors to focus on value-add • E.g., IBM, Avaki use GT3; Platform Globus • Open source is industry friendly!
DARPA, NSF, and DOE begin funding Grid work Globus Toolkit® History Does not include downloads from:NMI, UK eScience, EU Datagrid,IBM, Platform, etc. GT 2.0 Released Physiology of the Grid Paper Released Anatomy of the Grid Paper Released Significant Commercial Interest in Grids The Grid: Blueprint for a New Computing Infrastructure published NSF & European Commission Initiate Many New Grid Projects Early Application Successes Reported GT 1.0.0 Released NASA begins funding Grid work,DOE adds support
Managed shared virtual systems Computer science research Open Grid Services Arch Web services, etc. Real standards Multiple implementations Globus Toolkit Internet standards Defacto standard Single implementation The Emergence ofOpen Grid Standards Increased functionality, standardization Custom solutions 1990 1995 2000 2005 2010
Grid Communities • Global Grid Forum • Standards, information exchange, advocacy • 1000+ participants in tri-annual meetings • Application communities • E.g., physics, earthquake engineering, biomedical, etc. • Software development and support • NSF Middleware Initiative, UK eScience, Globus Toolkit, EGEE, …
Data Grids for High Energy Physics • Enable international community of 1000s to access & analyze petabytes of data • Harness computing & storage worldwide • Virtual data concepts:manage programs, data, workflow • Distributed system management
myGrid(Goble, De Roure, Shadbolt, et al.) • Imminent data deluge in bioinformatics • Heterogeneous, complex, and inter-related data sources • Integrated, community -wide treatment of data, literature, computational services
NEESgrid Earthquake Engineering Collaboratory U.Nevada Reno www.neesgrid.org
Distributed Aircraft Maintenance Environment (Austin et al.) In flight data Global Network eg: SITA Ground Station Airline DS&S Engine Health Center Maintenance Centre Internet, e-mail, pager Data centre
Manufacturing Financial Services LS / Bioinformatics Mechanical/ Electronic Design Process Simulation Finite Element Analysis Failure Analysis Other Derivatives Analysis Statistical Analysis Portfolio Risk Analysis Energy Entertainment Cancer Research Drug Discovery Protein Folding Protein Sequencing Web Applications Weather Analysis Code Breaking/ Simulation Academic Seismic Analysis Reservoir Analysis Digital Rendering Massive Multi-Player Games Streaming Media “Gridified” Infrastructure Industrial Perspective on Grids:A Wide Range of Applications Unique by Industry with Common Characteristics Grid Services Market Opportunity 2005 Sources: IDC, 2000 and Bear Stearns- Internet 3.0 - 5/01 Analysis by SAI
Open Infrastructure • Broadly deployed services in support of fundamental collaborative activities • Formation & operation of virtual organizations • Authentication, authorization, discovery, … • Services, software, and policies enabling on-demand access to critical resources • Computers, databases, networks, storage, software services,… • Operational support for 24x7 availability • Integration with campus and commercial infrastructures
Edinburgh Glasgow DL Newcastle Belfast Manchester Cambridge Oxford Hinxton RAL Cardiff London Soton Tier0/1 facility Tier2 facility Tier3 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link Open Infrastructure
Grid Communities & Technologies • Yesterday • Small, static communities, primarily in science • Focus on sharing of computing resources • Globus Toolkit as technology base • Today • Larger communities in science; early industry • Focused on sharing of data and computing • Open Grid Services Architecture • Tomorrow • Large, dynamic, diverse communities that share a wide variety of services, resources, data • Challenging computer science research issues
“The complexity of the problems facing mankind is growing faster than our ability to solve them. Finding ways toaugment our intellect is both a necessary & desirable goal.” Openness—in software, standards, and community—can be a powerful accelerator of progress Doug Engelbart Linus Torvalds New capabilities via resource sharing within distributed virtual organizations … … enabled by open software & standards within an international community. Summary: The Grid Explained,via the BCS Lovelace Medal