PRAGMA: Cyberinfrastructure, Applications, People

1. PRAGMA: Cyberinfrastructure, Applications, People Yoshio Tanaka (AIST, Japan) Peter Arzberger (UCSD, USA)

2. Science is an intrinsically global activity IVOA • The grid is transforming computing and • collaboration • The problem remains that the • grid is too hard to use • Middleware software needs to • interoperate PRAGMA’s Founding Motivations

3. PRAGMA Overarching Goals Establish sustained collaborations and Advance the use of the gridtechnologies for applications among a community of investigators working with leading institutions around the Pacific Rim Working closely with established activities that promote grid activities or the underlying infrastructure, both in the Pacific Rim and globally.

4. Affiliate Member PRAGMA PARTNERS

5. Key Activities and Outcomes • Encourage and conduct joint (multilateral) projects that promote development of grid facilities and technologies • Share resources to ensure project success • Conduct multi-site training • Exchange researchers • Advance scientific applications • Create grid testbeds for regional e-science projects • Contribute to the international grid development efforts • Increase interoperability of grid middleware in Pacific Rim and throughout the world Activities Outcomes

6. Contents:2004-2005 • Overview • Accomplishments • PRIME • Working Groups • Institutions • References • Opportunities • Sponsors http://www.pragma-grid.net

7. Working Groups: Integrating PRAGMA’s Diversity • Telescience – including Ecogrid • Biological Sciences: • Proteome Analysis using iGAP in Gfarm • Data Computing • Online Data Processing of KEKB/Belle Experimentation in Gfarm • Resources • Grid Operations center

8. Gfarm makes it possible to use iGAP to analyze the complete proteome (available 9/28/04) of the bacteria, Burkholderia mallei, a known biothreat agent, on distributed international resources. This is a collaboration under PRAGMA and the data is available through http://eol.sdsc.edu. A collaborative effort in the fight against bio-terrorism

9. Source: Cindy Zheng

10. PRAGMA Grid Testbed – overview – • Establish a regional wide testbed • Mainly focusing on computational Grid, later introduce other resources • Provides an easy-access environment for researchers, students, vendors, etc. • Testbed for software development and trial to have evaluation of usability and to archive performance numbers • Testbed for finding demonstrative / feasible applications • is going to be a production Grid

11. 言語と文字 PRAGMA Grid Testbed – unique features – • Truly (naturally) multi national/political/institutional VO beyond boundaries • Not an application-dedicated testbed – general platform • Diversity of languages, culture, policy, interests, … • Grid BYO – Grass roots approach • Each institution contributes his resources for sharing • Not a single source funded for the development • Physical resources • Most contributed resources are small-scale clusters • Networking is there, however the bandwidth is not enough • We can • have experiences on running international VO • verify the feasibility of this approach for the testbed development

12. Lessons Learned thru developments • Difficulties are caused by technical problems as well as sociological/political problems • Each site has its own policy • account management • firewalls • trusted CAs • … • Differences in interests • Application, middleware, networking, etc. • Differences in culture, language, etc. • Human interaction is very important

13. Building a Production Grid Thru Running Applications • Routine-basis experiments • Ninf-G based TDDFT, http://pragma-goc.rocksclusters.org/tddft/default.html • BioGrid, http://pragma-goc.rocksclusters.org/biogrid/default.html • iGAP over Gfarm • Learn requirements/issues • Research/implement solutions • Improve middleware integrations Source: Cindy Zheng

14. Status of Testbed in May, 2004 • Computational resource • 26 organizations (10 countries) • 27 clusters (889 CPUs) • Network performance is getting better. • Architecture, technology • Based on Globus Toolkit (mostly version 2) • Ninf-G (GridRPC programming) • Nimrod-G (parametric modeling system) • SCMSWeb (resource monitoring) • Grid Data FArm (Grid File System), etc. • Operation policy • Distributed management (No Grid Operation Center) • Volunteer-based administration • Less duty, less formality and less documents

15. Status of Testbed in May, 2004 (cont’d) • Questions??? • Ready for real science application? • Easy to use for every user? • Reliable environment? • Middleware stability? • Plenty document? • Enough security? and etc. • Direction of PRAGMA Resource Working Group • Do “Routine-basis Experiments” • Try daily application runs for a long term • Find out any problems and difficulty • Learn what is necessary for the production grid

16. Overview of Routine-Basis Exp. • Purpose • By daily runs of a sample application on PRAGMA testbed, find out and understand issues of the testbed operation for the real science application • Case study: the 1st application • Application • Time-Dependent Density Functional Theory (TDDFT) • Software requirements of TDDFT are Ninf-G, Globus and Intel Fortran Compiler. • Schedule • June 1, 2004 ~ August 31, 2004 (For 3 months) • Participants • 10 Sites (in 8 countries): AIST, SDSC, KU, KISTI, NCHC, USM, BII, NCSA, TITECH, UNAM • 193 CPUs (on 106 nodes)

17. Rough Schedule May June July Aug Sep Oct Nov 2 sites 5 sites 8 sites 10 sites SC’04 Setup Resource Monitor (SCMSWeb) 2nd user start executions 2nd App. start PRAGMA6 PRAGMA7 1st App. start 1st App. end • Summary of the experiment • Number of executions : 43 • Execution time • (Total) : 50.4 days • (Max) : 6.8 days • (Ave) : 1.2 days 1. Apply account 2. Deploy application codes 3. Simple test at local site 4. Simple test between 2 sites Join in the main executions after all’s done

18. Lessons Learned from the point of view of testbed operation • Need to reduce user’s effort • More documents • Need common configurations on each cluster • Should know the requirements of many applications • More collaboration among administrators from each site • Need to monitor site-to-site network Problem: Very unstable network between AIST and some sites • collaboration with networking community is important • e.g. APAN NOC • Need to monitor each cluster’s status Problem: Don’t know maintenance? trouble? network down? • SCMSWeb will help this issue. • Periodical status tests

19. Network Weather Map http://mrtg.koganei.itrc.net/mmap/grid.html Thanks: Dr. Hirabaru and APAN Tokyo NOC team

20. Summary • Dream big – build the imagination of those involved • Start small - build trust • Stay concrete – build infrastructure • Focus on people – build community • Opportunities: • Networking and Measurements • Testbed: Mutual learning • Applications: Geosciences, Astronomy, Ecology, Biomedicine, … • Exchange: Undergraduates and Graduates • Participate in Meetings and between meetings

21. Office of International Science and Engineering Division of Shared Cyberinfrastructure Division of Biological Infrastructure Thank you http://www.pragma-grid.net