1 / 37

ACAT2003 Session 1 Summary Talk: Computing Technology and Environment for Physics Research

This talk explores the evolution of computing platforms and the importance of Grid Services in distributed computing for physics research. Topics include advanced analysis environments, innovations in software engineering, and data management.

carmelc
Download Presentation

ACAT2003 Session 1 Summary Talk: Computing Technology and Environment for Physics Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ACAT2003Session 1 Summary Talk:Computing Technology and Environment for Physics Research Marcel Kunze Institute for Scientific Computing (IWR) Forschungszentrum Karlsruhe GmbH

  2. Topics (33 Talks and Round Table Discussion) • Parallel Computing Technologies and Applications (7 Talks) • Data Fabric and Data Management(5 Talks) • Online Monitoring and Control(3 Talks) • Advanced Analysis Environments(8 Talks) • Innovations in Software Engineering (5 Talks) • Graphic User Interfaces, Common Libraries(5 Talks) It is almost impossible to treat everything in this talk: It will be necessary to focus on specific topics.

  3. Computing Platforms Evolution At the Oberammergau workshop (1993), I had given a summary talk labeled „Computing in the 90‘s“. We saw the transition from mainframes to workstations and I emphasized the importance of the client-server model. Today, platforms and applications are far more • Advanced • Powerful • Dynamic • Complex This talk is about distributed computing and the importance of Grid Services.

  4. A little bit of History

  5. Grid Computing • Grid Computing has emerged as an important new field, distinguished from conventional distributed computing by ist focus on large-scale resource sharing, innovative applications, and, in some cases, high-performance orientation. Foster, Kesselman, Tuecke, The Anatomy of the Grid: Enabling Scalable Virtual Organizations, 2001 For us this means: • Provision of common tools, frameworks, environment, data persistency • Exploiting the resources available to experiments in computing centres, physics institutes and universities around the world • Presenting this as a reliable, coherentenvironment for the experiments • The goal is to enable the physicist to concentrate on science, unaware of thedetails andcomplexity of the environment they are exploiting

  6. The Golden Rule Keep it simple As simple as possible Not any simpler - Einstein

  7. Middleware • The tools that provide functions • that are of general application .. • …. not HEP-special or experiment-special • and that we can reasonably expect to come in the long term from public or commercial sources (cf internet protocols, unix, html)

  8. Grid Services

  9. Open Grid Services Infrastructure (OGSI) • Distributed applications are made of software components • Grid Services are an extension of Web Services • Discovery • Dynamic service creation • Lifetime management • Notification • OGSI defines a set of standardized interfaces and protocols • Currently available OGSI implementations • Unix: Globus Toolkit 3, OGSI::Lite (Perl), pyGridWare (Python) • Windows: OGSI.NET (Virginia Univ.); MS.NETGrid (EPCC) • GT3 has been evaluated by LCG (Talk: M. Lamanna) • Generally impressed with GT3 and the overall concept • GT3 IndexService: totally new, looks well designed • Information system and GRAM (critical parts of the GLOBUS kit) have problems of scalability and reliability

  10. Data Grids: Architecture • Replica Consistency Service in a Data Grid (Talk: G.Pucciani) • Replication of data increases system performance • Problem: Consistency management is an issue in applications where users can modify replicas • Integrating SRB with the GIGGLE/EDG framework (Talk: S.Metson) • Active collaboration between members of CMS, BaBar and the SDSC SRB Group • Problem: Could files stored in SRB be accessed by LCG tools? • Data discovery component is well understood • Full interoperation requires further development effort • Implementation of corresponding Grid services is planned

  11. Grid Tools: Monitoring • Configuration Monitoring Tool for Large Scale Distributed Computing (Talk: Y.Wu) • track and query site configuration information for large-scale distributed CMS applications • Plans to rework the tool as a Grid service

  12. Grid Tools: Monitoring • MonALISA: Monitoring Agents using a Large Integrated Services Architecture (Talk: I.Legrand) • Dynamic registration and discovery & subscription mechanism • Adaptability and self-organization

  13. Distributed Systems: Simulation • MONARC simulation framework (Talk: I.Legrand) • Modelling of large scale distributed computing systems • Design tool for large distributed systems • Performance evaluation

  14. Distributed Physics Data Analysis • Most HEP experiments are developing frameworks for distributed computing (M.Burgon-Lyon, G.Garzoglio:CDF,D0; P.Elmer,A.Hasan:BaBar; I.Adachi, G.Moloney:Belle; L.Taylor:CMS; A.J.Peters:ALICE) • Various workable solutions exist • Sometimes parallel and non-compatible effort • Importance of standardization • ARDA: Architectural Roadmap towards Distributed AnalysisRTAG11: http://www.uscms.org/s&c/lcg/ARDA/ • Common Grid analysis architecture for all LHC experiments • OGSI compliant • Concern: Analysis activities require chaotic access to resources by a large number of potentially inexperienced users (Professors) • Component-by-component deployment and avoiding big-bang releases are critical parts of the implementation strategy • Recommendation: Prototype based on AliEn (Talk: A.J.Peters)

  15. Advanced Analysis Environments

  16. Advanced Analysis Environments

  17. Advanced Analysis Environments

  18. General Re-Use of Components and Services (95%)

  19. Interactive Physics Data Analysis • Issues • Typical interactive requests will run on o(TB) distributed data • Transfer/replication times for the whole data about one hour • Data transfers once and in advance of the interactive session • Allocation, installation and set-up of corresponding database servers before the interactive session • Integration of user-friendly interactive access

  20. Interactive Physics Data Analysis

  21. Selection Parameters TagDB CPU Procedure PROOF DB1 RDB CPU Proc.C DB2 Proc.C DB3 CPU Proc.C DB4 CPU Proc.C DB5 CPU Proc.C DB6 CPU Parallel ROOT Facility: PROOF Local Remote Talk: Fons Rademakers

  22. PROOF: Actual Development

  23. PEAC System Overview

  24. Common Libraries: SEAL • SEAL Project Overview (Talk: L.Moneta) • SEAL has delivered basic foundation, utility libraries and object dictionary • The first version of the Component Model and Framework services is available • Scripting based on Python

  25. Common Libraries: PI • Physics Interface Project, Status and Plan (Talk: A.Pfeiffer) • Analysis Services components written in Python • Prototypes available to implement AIDA interface for HippoDraw and ROOT

  26. Graphic User Interfaces: QtRoot • Cross-platform approach to create the interactive application based on ROOT and Qt GUI libraries (Talk: V.Fine) • Qt package from TrollTech AS is a multi-platform C++ application framework that developers can use to write single-source applications that run-natively-on Windows, Linux, Unix, Mac OS X and embedded Linux. • A lot of Qt widgets available for re-use • Qt is the basis for the KDE desktop • Consolidation of Root Graphics(TGQt vs. TGWin32,TGX11,TGWin32GDK) Example: A fragment of STAR “Event Display” QtGLViewer class based viewer see: http://www.rhic.bnl.gov/~fine/EventDisplay )

  27. Fabric Area

  28. Data Fabric and Data Management • Need for powerful, high throughput systems • Storage Area Networks • GridKa scalable IO design based on fibre channel technique(Talk: J.van Wezel) • Infiniband yields 800 MB/s (Talk: U. Schwickerath) • Need for powerful trigger systems to reduce data • Realtime analysis for the ALICE HLT (Talk: C.Loizides) • Need for powerful clusters and networks for online event reconstruction and distributed analysis • Realtime event reconstruction farm for Belle (Talk: R.Itoh) • A basic R&D for an analysis framework distributed on wide area network (Talk: H. Sakamoto) • New methods for data integration and management • Grid portal based data management for lattice QCD data (GENIUS Talk: G.Andronico)

  29. Round Table Discussion D. Laforenza

  30. It took 200 Years to develop electrical Grids

  31. Open Questions • Is the far-reaching vision offered by Grid Computing obscured by the lack of interoperability standards among grid computing technologies ? • Should the next few years be considered as a transition period with multiple prototypes in competition to speed up the development ?

  32. How to design Grid-aware Applications? • Make developers and users aware of network based applications • Need to think about new abstract programming models • Development of new programming techniques and tools that specifically address the Grid and encompass • Heterogeneity • Distributed computing aspects of Grid programming

  33. CrossGrid: Tools for easy Use of the Grid

  34. CrossGrid: Migrating Desktop • Idea • Save and resume a user grid session • Look and feel of a windows desktop • Implementation • Roaming Access Server and Clients • Java Web Services (Portability) • Integration of Tools • Job submission wizard • Job monitoring dialog • GridExplorer dialog • GridCommander dialog

  35. Outlook • Scaling of Fabric Infrastructure • Cheap commodity components vs. High-tech solutions (e.g. SAN) • Note: Each service needs an operator • Total cost of ownership has to take into account infrastructure & manpower • What will be the business model for the Grid market place of resources ? Unlimited access ? Credit points ? Cash ? • ARDA prototype will push development of Physics applications in a distributed environment • What will the production environment look like? • Components will be based on Grid Services • Open Grid Service Infrastructure is the common denominator • Rapid prototyping and user feedback is essential ! • Concern: Users only change their paradigm of working if they see added value (better results, faster turn-around, additional resources etc.) !

More Related