1 / 27

International Computing e-Infrastructures: Past, Present and Future...

International Computing e-Infrastructures: Past, Present and Future. Fabrizio Gagliardi EMEA Director Technical Computing Microsoft Corporation. Some personal introductory remarks Definition of e-Infrastructures Need for e-Infrastructures Recent past history

bert
Download Presentation

International Computing e-Infrastructures: Past, Present and Future...

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. International Computing e-Infrastructures: Past, Present and Future... Fabrizio Gagliardi EMEA Director Technical Computing Microsoft Corporation

  2. Some personal introductory remarks Definition of e-Infrastructures Need for e-Infrastructures Recent past history Current situation, accomplishments and challenges Outlook for the future Outline of the talk CGW'05

  3. Infrastructures to support wide geographically distributed communities which share problems and resources to work towards common goals Leveraging international network interconnectivity Based on safe AAA architecture Need persistent software middleware (S/W is integral part of the infrastructure) Definition of e-Infrastructures CGW'05

  4. ‘Grids’: A Catch-All Marketing Term • ‘Grids’ mean many different things to many different people/companies: • P2P desktop cycle-stealing • Linked Supercomputer Centers • Managed virtual distributed clusters • Internet access to giant, distributed repositories • Virtualization of data center IT resources • Out-sourcing to “utility compute centers” • Sharing resources distributed among different administrative domains (Ian Foster) • For Microsoft, Grids are about Data Management as much as Compute Cycles CGW'05

  5. Science, industry and commerce are more and more digital, process vast amounts of data and need massive computing power We live in a “flat” world: Science is more and more an international collaboration and often requires a multidisciplinary approach Need to use technology for the good cause Fight Digital/Divide Industrial uptake has become essential Need for e-Infrastructures CGW'05

  6. Meta-computing and distributed computing early examples in the 80’ and 90’ (CASA, I-Way, Unicore, Condor etc.) EU-US workshop in Annapolis in 1999 on large scientific data bases: http://www.cacr.caltech.edu/euus/ EU FP5 and US Trillium and national Grids EU FP6, US OSG, NAREGI/Japan… Recent past history CGW'05

  7. Chronology IGO-----ETICS/EGO---- EDG start 2001 EGEE-II start 2006 CHEP2000 EGEE-XX 2008? EGEE start 2004 We are here CGW'05

  8. ARCADE 2002 Barcelona, 14th February 2002 “Unlimited” bandwidth, breaking the frontiers of computing: the path in Europe from FP5 to FP6. Antonella Karlson Research Networks Unit, DG INFSO, EC antonella.karlson@cec.eu.int CGW'05 "The views expressed in this presentation are those of the author and do not necessarily reflect the views of the European Commission"

  9. EGSO Applications GRIA CROSSGRID EUROGRID GRIP DATAGRID Middleware& Tools GRIDLAB GRIDSTART DAMIEN DATATAG Underlying Infrastructures Industry / business Science GRIDs - IST projects (~36m Euro) An integrated approach CGW'05

  10. DATAGRID, CROSSGRID GEANT INFRASTRUCTURE • Application requirements: • Computing > 20 TFlops/s • Downloads > 0.5PBytes • Network speeds at 10 Gbps • 17 European countries • Collaboration of more than 2000 scientists GRIDs: Examples of large testbeds CGW'05

  11. DATATAG (cross-Atlantic testbed) (2Gbps) Links with US projects (GriPhyN, PPDG, iVDGL,…) GRIDs: Examples of large testbeds CGW'05

  12. Many Grids around the world, very few maintained as a persistent infrastructure (best example is the “secret” Google Grid) Need for public and open Grids (OSG, EGEE and related projects, NAREGI, and TERAGRID, DEISA good prototypes) Persistence, support, sustainability, long term funding, easy access are the major challenges Current situation: accomplishments and challenges CGW'05

  13. Projects in Europe (I) • Access to IT-resources (connectivity, computing, data, instrumentation…) for scientists: • Providing e-Infrastructure • Géant2 • EGEE • DEISA • SEE-GRID • Benefiting from e-Infrastructure • DILIGENT • SIMDAT • GRIDCC • CoreGRID • GridLab • Concertation: GRIDSTART, GridCoord • Grid mobility: Akogrimo CGW'05

  14. D-GRID Projects in Europe (II) • Sample of National Grid projects: • Austrian Grid Initiative • DutchGrid • France: • e-Toile • ACI Grid • Germany • D-Grid • Unicore • Grid Ireland • Italy • INFNGrid • GRID.IT • NorduGrid • UK e-Science • National Grid Service • OMII • GridPP project CGW'05

  15. Policy Forums • The e-Infrastructures Reflection Group (eIRG) • Mission: study and promote policies for easy and cost-effective shared use of electronic resources in Europe • 25 countries (government-appointed representatives), EU: 2 members • White Papers • European Strategy Forum on Research Infrastructures (ESFRI) • Role: to support a coherent approach to policy-making on research infrastructures in Europe, and to act as an incubator for international negotiations about concrete initiatives • representatives of the 25 EU Member States, appointed by Research Ministers and a representative of the European Commission • ESFRI + eIRG: European roadmap for new research infrastructures of pan-European interest (10-20 years) CGW'05

  16. Géant2 • GÉANT2 is the 7th generation of the pan-European research and education network, successor to the multi-gigabit research network GÉANT. • Official start: 1 September 2004, Duration: 4 years • Funding: EC, national research, education networks • Managed by DANTE • Goal: • To connect 34 countries through 30 national research and education networks (NRENs) • using multiple 10Gbps wavelengths • Status: • Equipment and services currently in operations (officially inaugurated by Commissioner Reding last June) • Transition from GÉANT network to GÉANT2 gradually completing, started in the first quarter of 2005 CGW'05

  17. GRIDCOORDBuilding the ERA in Grid research K-WF GridKnowledge basedworkflow & collaboration inteliGRIDSemantic Grid based virtual organisations Grid-based generic enabling application technologies to facilitate solution of industrial problemsSIMDAT OntoGridKnowledge Services for the semantic Grid UniGridSExtended OGSAImplementation based on UNICORE EU-driven Grid services architecture for businesS and industry NextGRID Mobile Grid architecture and services for dynamic virtual organisations Akogrimo DataminingGridDataminingtools & services HPC4UFault tolerance,dependabilityfor Grid European-wide virtual laboratory for longer term Grid research-creating the foundation for next generation Grids CoreGRID ProvenanceTrust and provenance for Grids Specific support action Integrated project Network of excellence Specific targeted research project EU Grid technology & infrastructure (I) New Grid Research Projects in FP6 EU Funding:52 MILLION - Start: SUMMER 2004 From a talk by Ulf Dahlsten, Den Haag, Nov 2004 CGW'05

  18. GÉANT network GÉANT network (FP6) TEN 155 network TEN 155 network IPv6 testbeds IPv6 testbeds IPv6 actions 2000 2001 2002 2003 2004 2005 2006 2007 2008 Grid testbeds Grid enabled Infrastructures (EGEE, DEISA, SEE-GRID,…) (other) testbeds (other) testbeds FP5 FP6 Complementary to National infrastructures EU Grid technology & infrastructure (II) Building the European eInfrastructurefor research FP7 From a talk by Ulf Dahlsten, Den Haag, Nov 2004 CGW'05

  19. Real time Grid for remote control of instruments User involvement… Optical solutions for Grid infrastruct. …technology validation New user communities using Grids – Digital Libraries Flexible Quality of Service Assurance Under negotiation GRIDCC MUPPET DILIGENT EUQoS EGEE DEISA SEE-GRID eInfrastructure IPv6TF SC EUROLABS IPv6 Task Force support Experimental testbeds LOBSTER Specific Support Actions Traffic monitoring EU Grid technology & infrastructure (III) eInfrastructure – Testbeds Courtesy of K. Baxevanidis, EU CGW'05

  20. EU Grid technology & infrastructure (V) • eInfrastructure – achievements • Connectivity service • Computing, storage service • Testbeds • International links • GÉANT network: 10Gbit/s, IPv6 enabled, 3900 Research Centres connected • EGEE: production quality, >10000 CPUs, >5PB storage, training, coverage of 27 countries • DEISA: Supercomputer network, reaching 40 Tflop/s • Rich set of technologies tested/ verified (IPv6, Grids, Optical, End-to-End QoS, Security, Mobility…) and communities involved (scientific, industry) • USA, Russia, Mediterranean, Asia, Latin America... From a talk by Ulf Dahlsten, Den Haag, Nov 2004 CGW'05

  21. Outlook for the future Outlook for the future CGW'05

  22. Supercomputing Goes Personal CGW'05

  23. The Continuing Trend Towards Decentralized, Networked Resources Grids of personal & departmental clusters Personal workstations & departmental servers Minicomputers Mainframes CGW'05

  24. Parallel applications development High-productivity IDEs Integrated debugging/profiling/tracing/analysis Code designer wizards Concurrent programming frameworks Platform optimizations Dynamic, profile-guided optimization New programming abstractions Distributed systems issues Web Services & HPC grids Security Interoperability Scalability Dynamic Systems Management Self (re)configuration & tuning Reliability & availability RDMS + data mining Ease-of-use Advanced indexing & query processing Advanced data mining algorithms Digital experimentation Collaboration-enhanced Office productivity tools Structure experiment data and derived results in a manner appropriate for human reading/reasoning (as opposed to optimizing for query processing and/or storage efficiency) Enable collaboration among colleagues (Scientific) workflow environments Automated orchestration Visual scripting Provenance Leverage IT Industry’s Existing R&D CGW'05

  25. Past Buy lab equipment Keep lab notebook Run experiments by hand Assemble & analyze data (using stat pkg) Collaborate by phone/email; write up results with Latex Metaphor: Physical experimentation “Do it yourself” Lots of disparate systems/pieces Future Buy hardware & software Automatic provenance Workflow with 3rd party domain packages Excel & Access/SQL-Server Office tool suite with collaboration support Metaphor: Digital experimentation Turn-key desktop supercomputer Single integrated system Scientific Information Worker:Past and Future CGW'05

  26. Like in the past ES, AI, networking, OS they will disappear from the hot research (and hype) space and become mainstream technology Major Grids already work in production (EGEE: 18’000 computers, Google: 100’000 computers?...) Major IT vendors will integrate Grid middleware in their standard products (industrial uptake) Computing and data resources will become commodities on the Internet ISPs will offer a wide range of services Grid based, a full mature market will develop for these services The result will be a tremendous computing and data processing power which will enable a new set of scientific applications and generate large revenues for business applications A potential leveler for a worldwide science and economy => digital Divide could be moderated Where Grids will be in 5 years? CGW'05

  27. … And time will tell how wrong we are in our predictions now See you back here next year! CGW'05

More Related