TeraGrid Software Integration: Area Overview (detailed in 2007 Annual Report Section 3)

1. TeraGrid Software Integration:Area Overview(detailed in 2007 Annual Report Section 3) Lee Liming, JP Navarro TeraGrid Annual Project Review April, 2008

2. Software Integration (SI) Activities • GIG Software Integration • Area Coordination • Operations • Define and maintain six CTSS capability definitions • CTSS software packaging • Operate infrastructure services • Development • Information Services • TeraGrid Client Software Distribution • Application Hosting Prototype • Instrumentation • Resource Providers • CTSS Deployment & Maintenance • RP Capability Development Software Integration Overview

3. Coordinated TeraGrid Software & Services (CTSS) What is CTSS? The collection of capabilities that TeraGrid resource providers coordinate across their compute and storage resources Examples: • Remote job submission interface • Compilers, some development libraries and tools • Data movement CTSS is NOT: • All the software available across all TeraGrid resources • Software used in centralized TeraGrid services (User Portal, TeraGrid website, TeraGrid central accounting and allocations, gateway software) • User, application, or community software History: CTSS 1: DTF construction and initial production common software and services (2003) CTSS 2: ETF common software and services (2004) CTSS 3: Incremental evolution (with Globus 4) common software and services (2006) CTSS 4: A new paradigm (2007) Software Integration Overview

4. CTSS 4 Capability Kits • Availability: • First kit (Core) is required and available everywhere • Next 6 kits are available (almost) everywhere • Last 2 kits are available on a subset of resources Software Integration Overview

5. CTSS 4 – A New Paradigm • Significant change in how we define CTSS • CTSS 1 thru 3: Monolithic software stack • CTSS 4: Modular user capabilities • Improved many aspects of capability delivery • Better descriptions of the capabilities (esp. for users) • Better documentation • Clearer availability information • More focused delivery process (package, deploy, and configure) • Improved process for RPs to select and publish their offerings • Delivery timeline • Designed in 2006 • Capability kits defined in Q1-Q2 2007 • Capabilities rolled out in Q2-Q3 2007 Software Integration Overview

6. Motivation for a Paradigm Shift • Transform CTSS from a complex puzzle of interlocking pieces to a menu of recognizable user functions • Improve agility in response to new user requirements • Easier for users to discover and learn about capabilities • Operational benefits • Improves delivery scalability thru distributing ownership • Streamlined change management process • Makes it possible to evaluate and adopt competitors to CTSS components (or eliminate existing components) • Easier to correlated component usage data to specific user benefits and types Software Integration Overview

7. Information Service • The linchpin in TeraGrid’s coordinated user environment • How resource providers register availability of CTSS capabilities • How users discover coordinated capabilities and how to access them • Integrated with user documentation, user portal, Inca, gateways • GIG contributions • Architecture and technology selection • Information schema • New information providers • Presentation views and services • High-availability central information service • RP contributions • Local information services • Data! See section 3.3 of annual report Software Integration Overview

8. Software Integration Challenges • Expected issues with integrating diverse high-end systems • Integrating with local RP specialized services • Integrating with local RP security policies/practices • Meshing with local resource hardware and software operations schedules • Scalability and reliability issues in specific capabilities • LEAD gateway; NCSA, IU, UofC/ANL RPs; Globus team; PSC End-to-End data movement group; and GIG Gateways and Software Integration teams working for several months to address operational, configuration, and software issues with Grid services. • Human Arterial Tree simulation team; IU RP; MPICH-G2/mpiG team; and GIG SI team worked to resolve deployment and customization issues in software. • Complexity in new capabilities • New Remote Compute version 4 capability will integrate 5 distinct web services, and 1 non-web service to deliver functionality Software Integration Overview

9. Operations Activities • Coordinate the CTSS capability definition process • Document the capabilities – user scenarios and key interfaces • Define/maintain a recommended implementation • Track user requirements • Track/address software component issues • Includes formal change management process • Maintain six of the CTSS capability definitions • Generate CTSS software packages • Build binaries and package 11 CTSS components (9 with OSG’s VDT) • Package 24 other CTSS components • Why? Security vulnerabilities, software issues, RP technical requirements, new resources and platforms, new operational requirements • Operate GIG-provided services Software Integration Overview

10. Capability Development Activities • TeraGrid client software distribution • Application hosting capability • Usage reporting • Build and test – Metronome service • Fostering interoperability with other major grids • Fostering international grid standards • Several development projects at IU, NCAR, NCSA, PSC, TACC, UC/ANL Software Integration Overview