Distributed Computing Instrastructure as a Tool for e-Science

1. Distributed Computing Instrastructure as a Tool for e-Science Jacek Kitowski, Kazimierz Wiatr, Łukasz Dutka, Maciej Twardy, Tomasz Szepieniec, Mariusz Sterzel, Renata Słota and Robert Pająk ACK Cyfronet AGH PL-Grid Consortium PPAM 2015, 7-9.09.2015, Kraków

2. Outline • National e-Infrastructure • Assumptions and foundations • Tool for e-Science • e-Infrastructurecreation – motivation, background and issues • Conceptualization and implementation • PLGridcasestudy • Enhancement of Achievements • Platforms and Environments – SelectedExamples • Conclusions

3. e-InfrastructureCreation Motivation and Background • Increasing importance of Computational Science and Big Data Analysis • Needs: • Preventing users from technical problems • Expert support for making science • Increase of resources • Openess for future paradigms Numericallyintensivecomputing • Experiments in silico: • Computing and Data for Open Science • International collaboration • User/platform driven e-infrastructure innovation (e-Science and e-Infrastructure interaction) Data intensivecomputing • Computational Science problems: • Algorithms, environments and deployment • Future and emergingtechnologies • 4thparadigm, distributed, grid and cloudcomputing,Data Farming • Activity initiated by Cyfronet

4. e-InfrastructureCreation Issues Synergistic effort in several dimensions: • Meeting user demands in the field of grand challenges applications • Activity supported by users with scientific achievements and by well-defined requirements Meeting user demands • Organizational: • horizontal perspective - federation of computer centres supporting thee-infrastructure with different kinds of resources and competences • vertical perspective - involvement of computer, computationaland domain-specific experts intoe-infrastructure operations Energy awareness Organizational Technological • Technological: • different computinghardware and software • variousmiddlewareenvironments • Energy awareness: • optimal scheduling strategies of computing jobs among federationresources to minimize energy consumption as a whole

5. PL-GridConsortium • Consortiumcreation – 2007 • a response to requirements from Polish scientists • due to ongoing eScience activities in Europe and in the World • Aim:significantextension of computing resources and solutionsprovided to the scientific community • PL-GridProgramme • Development based on (SWOT analysis): • projectsfunded by the EuropeanRegional Development Fund as part of the InnovativeEconomy Program • closeinternationalcollaboration (EGI, ….) • previousprojects (5FP, 6FP, 7FP, EDA…) • National Network Infrastructure available: Pionier National Project • computingresources: Top500 list • Polish scientific communities: ~75% highly rated Polish publications in 5 Communities PL-Grid Consortium members: 5 High Performance Computing Polish Centres, representing the Communities coordinated by ACC Cyfronet AGH

6. High Performace Computing Centre of Competence High Performance Networking ACK Cyfronet AGH42 years of expertise HumanResources Social Networking Infrastructure Resources Network Resources

7. The most powerful HPC Asset(in Poland) • Prometheus Cluster (2014/2015) • Rpeak= 1658.9 TFlops • 1728 servers • 41,472 Haswellcores • 216 TB RAM (DDR4) • 10 PB disks, 180 GB/s • HP Apollo 8000 • In operationApril 2015 • Q4 2015 Extensions • Rpeak= 483.8 TFlops • 504 servers • 12,096 Haswellcores • RpeakNVIDIA= 256.3 TFlops • 144 Nvidia K40 XL • In SUMMARY: 2,4 PFlops (with GPU) 49th position on the July 2015 edition of the TOP500 list

8. TOP500, July 2015 Polish Sites

9. Family of PL-Grid Projects coordinated by Cyfronet • PL-Grid (2009–2012) • Outcome: Common base infrastructure Assumed Performance • PLGrid PLUS (2011–2015) • Outcome: • Focus on users (training, helpdesk…) • Domain specific solutions: 13 +1500 Tflops +500 Tflops • PLGrid NG (2014–2015) • Outcome: • Optimization of resourcesusage, training • Extension of domainspecific by14 +8 Tflops Real Users 230 Tflops • PLGrid CORE (2014–2015) • Outcome: Competence Center • Open Science paradigm(large workflow app., data farming mass comp., ……) • End-user services

10. Summary of ProjectsResults (up-to-date) • Close collaborationbetween Partners and researchcommunities • Development of tools, environments and middleware services, Clouds • Integration, HPC, Data intensive, Instruments • Development of 27 domainspecificsolutions • Development of IT PL-GridInfrastructure and ecosystem

11. Summary of ProjectsResults (up-to-date) • 26paperson PL-Grid Project results • Facilitation of community participation in international collaboration • EGI Council, EGI Executive Board • FP7 (VPH-Share, VirtROLL….) • EGI-InSPIRE, FedSM, … • EGI-Engage, Indico DataCloud, EPOS, CTA, PRACE, H2020…. • Publications • 36paperson PLGrid Plus Project results147 authors, 76 reviewers

12. Journal Publications (subjectiveselection) Conferences: • CracowGrid Workshop (since 2001) • KU KDM (since 2008)

13. Summary of ProjectsResults (up-to-date) # users’ grants (active) # users

14. Summary of ProjectsResults (up-to-date) • Examples of activegrants • PROTMD (18.9.2015-18.9.2016) – CyfronetResearch on proteinsusing MD25 mln hours (2,800 cores) • PCJ2015GA (26.8.2015-31.12.2015) – ICMResearch on connectome of nematodesusing GA15 mln hours (6,000 cores) • PSB (1.3.2015-1.3.2016) – TASK, Cyfronet, ICMM, WCSSNew characteristics of DNA in the context of tumor therapy11 mln hours (1,200 cores)

15. Summary of ProjectsResults (up-to-date)

16. Deployed PLGrid IT Platforms and Tools – selected examples (by Cyfronet)

17. GridSpaceA platform for e-Science applications • Experiment: an e-science application composed of code fragments (snippets), expressed in either general-purpose scripting programming languages, domain-specific languages or purpose-specific notations. Each snippet is evaluated by a corresponding interpreter. • GridSpace2 Experiment Workbench: a web application - an entry point to GridSpace2. It facilitates exploratory development, execution and management of e-science experiments. • Embedded Experiment: a published experiment embedded in a web site. • GridSpace2 Core: a Java library providing an API for development, storage, management and execution of experiments. Records all available interpreters and their installations on the underlying computational resources. • Computational Resources: servers, clusters, grids, clouds and e-infrastructures where the experiments are computed. Contact: E. Ciepiela, D. Harężlak, M. Bubak

18. InSilicoLab science gateway framework • Goals • Complex computations done in non-complex way • Separating users from the concept of jobs and the infrastructure • Modelling the computation scenarios in an intuitive way • Different granularity of the computations • Interactive nature of applications • Dependencies between applications • Summary • The framework proved to be an easy way to integrate new domain-specific scenarios • Even if done by external teams • Natively supports multiple types of computational resources • Including private resources – e.g. private clouds • Supports various types of computations Architecture of the InSilicoLab framework: Domain Layer Mediation Layerwith its Core ServicesResource Layer with differentkinds of workers Different kinds of users  different kinds of resources Contact: J. Kocot, M. Sterzel, T. Szepieniec

19. DataNetcollaborativemetadatamanagement Objectives • Provide means for ad-hoc metadata model creation and deployment of corresponding storage facilities • Create a research space for metadata model exchange and discovery with associated data repositories with access restrictions in place • Support different types of storage sites and data transfer protocols • Support the exploratory paradigm by making the models evolve together with data Architecture • Web Interface is used by users to create, extend and discover metadata models • Model repositories are deployed in the PaaS Cloud layer for scalable and reliable access from computing nodes through REST interfaces • Data items from Storage Sites are linked from the model repositories Contact: E. Ciepiela, D. Harężlak, M. Bubak

20. Onedatatransparent access to data A system that provides a unified and efficient access to data stored in organizationally distributed environments. Onedata Global Registry • Provides a uniform and coherent view on all data stored on the storage systems distributed across the infrastructure • Supports working in groups by creation of an easy-to-use shared workspace for each group. • Servesdata efficiently Contact: Ł. Dutka

21. Scalarmdata farming experiments Scalarmoverview What problemsare addressed with Scalarm ? Data farming experiments with an exploratory approach Accessing heterogeneous computational infrastructure 75% allsubmittedtasks Self-scalable platform for parametricstudies Adapting to experiment size and simulation type Exploratory approach for conducting experiments Supporting online analysis of experiment partial results Integrates with clusters, Grids, Clouds Parameter space generation with support of design of experiment methods Self-scalability of the management/execution parts Scalarm Graphical User Interface Contact: R. Słota

22. Rimrockaccess to resources Aservice which simplifies the management of processes andtasks executed in the PLGridinfrastructure. Rimrockarchitecture Rimrockfeatures simplicity – non-complicated integration with other applications, scripts and services interactivity – a user can modify working processes based on indirect results universalism – supported by many programming languages versatility – it allows to execute an application in a batch mode or startan interactive application user friendliness – it does not require advanced knowledge (basic information about Bash shell and curl command are sufficient to start using it) Contact: D. Harężlak

23. Cloud Computing • The Cloud increases elasticity of research, as scientists can tune the virtual machines to their specific needs. • The catalogue of VMs offered by PL-Grid contains many OSs. • Cloud platform is also the best and in many cases the only solution for running jobs with legacy software packages. • Open Nebula migration to Open Stack, …. • Cloud Platform for VPH-Shareapplications (Atmoshereenv.) • IaaS, PaaS, STaaS…. Contact: J. Meizner, T. Szepieniec, M. Radecki

24. Cloudenvironment for VPH-Shareapp. Portal and Atmosphere

25. Applications Catalog service • Objective: • to present in one place and in a uniform manner the current offer of the software available in the PLGrid infrastructure, broken down into supercomputing centers, clusters as well as categories and areas of application. Applications Catalog is a system collecting and providing information on the applications, development tools and libraries offered in the PLGrid infrastructure. It allows to search for applications, check the status of their operation, obtain information about changes and updates, as well as it provides documentation and examples of usage. It is designed for all those interested in the use of the applications available in the PLGrid infrastructure.

26. Map-Reduce service • Apache Spark 1.5.0 functionality: • API, RDD, DataFrame, SQL • Backend Execution: DataFrame and SQL • Integrations: Data Sources, Hive, Hadoop, Mesos and Cluster Management • R Language • Machine Learning and Advanced Analytics • Spark Streaming

27. Summary and Conclusions • Three dimensions of development: • HPC/GRID/CLOUDs • Data & Knowledge layer • Network & Future Internet • Deploymentshave the nationalscope; however with closeEuropeanlinks • Development oriented on end-users & researchprojects • Achievingsynergybetweenresearchprojects and e-infrastructuresby closecooperation and offeringrelevant services • Durabilityatleast 5 yearsafterfinishing the projects - confirmed in contracts • Futureplans: continuation of development • Center of Excellence • CGW, KUKDM as places to exchange experienceand for collaboration between eScience centers in Europe

28. More information • Please visit our Web pages: • http://www.plgrid.pl/en • http://www.plgrid.pl • CREDITS!

29. Credits • ICM • Marek Niezgódka • Piotr Bała • Maciej Filocha • PCSS • MaciejStroiński • Norbert Meyer • Krzysztof Kurowski • Tomasz Piontek • PawełWolniewicz • WCSS • Jacek Oko • Józef Janyszek • Mateusz Tykierko • Paweł Dziekoński • Bartłomiej Balcerek • TASK • Rafał Tylman • Mścislaw Nakonieczny • Jarosław Rybicki • ACC Cyfronet AGH • MichałTurała • Marian Bubak • Krzysztof Zieliński • Karol Krawentek • AgnieszkaSzymańska • Maciej Twardy • Angelika Zaleska-Walterbach • Andrzej Oziębło • ZofiaMosurska • Marcin Radecki • RenataSłota • Tomasz Gubała • Darin Nikolow • Aleksandra Pałuk • PatrykLasoń • Marek Magryś • ŁukaszFlis Special thanks to manydomainexperts ! … and manyothers…..