OGF eScience and OGF Software

1. OGF eScience and OGF Software Geoffrey Fox GFSG Meeting Kilburn Building Manchester University May 6 2007

2. Role of Software and Standards in Grid Adoption I Grids are increasingly being deployed around the world using a variety of open source and commercial software platforms, modules and capabilities – many of which are based on proprietary interfaces with support for only the most broadly adopted standards. Users primarily adopt software solutions, NOT standards whether from OASIS, W3C or OGF Standards come to attention of software providers when the lack of standards creates a highly visible “pain point” or barrier to effective utilization. Software providers are often skeptical of all but the most broadly adopted standards that have already proven their desirability within the market. These providers are encouraged by the market to focus on practical, user driven demands and must continually tradeoff: (1) fixing bugs; (2) adding new features; (3) adopting standards – thus setting a high bar for incorporating standards within their demanding release schedules.

3. Role of Software and Standards in Grid Adoption II Looking at deployed Grids, one finds the simple Web service standards (WSDL, SOAP, XML) are critical but the more complex standards such as WS-RM and even BPEL and WS-Security have limited adoption. Rather software modules such as Condor and SRB set de facto interface points. (http://grids.ucs.indiana.edu/ptliupages/publications/ReviewofServicesandWorkflow-IU-Aug2006B.pdf) Web 2.0 shows the same trend in more extreme fashion with Google maps exemplifying a hugely popular module with no adherence to either Web Service or Open Geospatial Consortium standards. HTTP is the Web 2.0 universal standard and the field suggests that one generates broader adoption by focusing on simplicity and ease of entry for developers rather than by specified interoperability. Why can Web 2.0 get away with such disdain for standards. It is trading off advantages of rich protocols used in WS-*/Grids versus advantages of low barrier to entry. Presumably the approach that leads to the “best” user experience will be adopted and this could imply different technology choices in different application areas.

4. Role of Software and Standards in Grid Adoption III The OGF supports broader exploitation and deployment of Grids with a mix of standard setting and community activities. Often OGF processes are aimed at driving user requirements all the way to relevant standard specifications. Other activities such as workshops and software forums promote presentations and discussions that both inform about and advance the state of the art. Both community and standards activities advance OGF’s broad goal but their different capabilities and roles are not well understood and integrated. For example software providers should be interested in both standards and interacting with users but it is not clear that user-standards interactions are especially fruitful. These observations suggest that it be useful to debate the roles of standards and software and their interaction together with positioning of user, software provider and standards communities. How should OGF position its activities to maximize the “broader exploitation and deployment of Grids”?

5. Web 2.0 APIs • http://www.programmableweb.com/apis has (April 17 2007) 415 Web 2.0 APIs with GoogleMaps the most often used in Mashups • This site acts as a “UDDI” for Web 2.0

6. The List of Web 2.0 API’s • Each site has API and its features • Divided into broad categories • Only a few used a lot (39 API’s used in more than 10 mashups) • RSS feed of new APIs

7. Web 2.0 API’s versus Protocol

8. Four major types of eScience Activities • Group activities such as those in GIN (Interoperability) • Groups perform long term activities in focused areas • Typically does not get much high level visibility except for GIN • Note two newish groups at OGF19 in Education and Reliability • Typically one or more distinct 90 minute sessions • Timely community activities arranged in the two-five months before meeting and including panels, tutorials and short workshops • Current submissions to community program • Typically one or two 90 minute sessions per submission • Long lead time single track workshops with invited and contributed presentations in topics of broad interest to OGF. • Typically 4 or 5 90 minute sessions • Refereeing similar to high quality conferences • Software Development Track starting at OGF19 • 14 90 minute sessions at OGF19 8

9. Community Program Users Grid Projects • Community oriented activities with relatively light weight approval process with call 3-5 months before meeting and decisions 2-3 months before • Joint between Enterprise and eScience • Exploratory (Birds of a Feather) sessions that could leads to groups or full one-day workshops • Tutorials but not well done in OGF as no easy thoughtful coordination and don’t easily attract Grid users (as opposed to Grid builders) • Need to establish a track with a uniform audience like we propose for software development • Community workshops often small (2 sessions) and led by groups such as OGF20 Astro-RG/PE-RG/GINThe Astronomical Virtual Observatory - Building Operational Services on Pervasive Grids: Standards in Use Grid Subsystems Grid Services 9

10. Software Development Sessions Users Grid Projects • Grids are built from services which hopefully respect standards • However there are several building blocks or subsystems like Globus or SRB which are used in many Grid projects • TeraGrid EGEE Geon LEAD MyGrid China National Grid Naregi are Grid Projects • Form initially at least de facto standards • What is OGSA in the “non Green Field” of existing subsystems? • OGF will offer “user group” sessions in “Grid Subsystems” in a set of consecutive sessions aimed at those building Grids and not at people using Grids • OGF can provide one-stop shopping so don’t need to attend XYZweek for all XYZ • GIN-Standards Interaction? Grid Subsystems Grid Services 10

11. OGF19 Software Development Track • Clarens Grid Portal Toolkit • Condor Scheduling system • Genesis II OGSA Grid Infrastructure • Globus core Grid Infrastructure • Grid Federated Identity (GridShib, GAARDS, MyProxy) • GridSphere portlet container for portals • Ninf-G core Grid RPC Infrastructure • NWS and BQP Network/Queuing Tools • OGCE Open Grid Computing Environments collection of portlets (for Science Gateways) • OMII core Grid infrastructure (includes OGSA-DAI and Taverna) • SRB data Grid infrastructure • Unicore core Grid infrastructure 11

12. Future of Software Development Track Not clear how many at OGF20 but not very many! Less widely adopted software could be showcased in community program Suggested additional software from call – no response to request to select from these A Introduce/GAARDS: Suggested by Stephen A. Langella langella@bmi.osu.edu (part of development team) Action: GAARDS already part of track in federated identity session organized by Von Welch B: Nimrod Tools for Distributed Parametric Modeling: Suggested by Geoffrey Fox to broaden geographical scope C: CGSP China Grid Supporting Platform: Suggested by Geoffrey Fox to broaden geographical scope D: GOS China National Grid Software Environment: Suggested by Fox to broaden geographical scope E: European Grid Software: Suggested by Mirco Mazzucato (INFN Padua, Italy) and others EGEE core grid Infrastructure , EGEE high level services , SRM based MSS , INFN Grid policy and accounting framework F: Nordugrid core grid infarstructure: Suggested by Jean-Pierre Prost (IBM, France) and Balazs Konya (Lund University, Sweden) G: ProActive (http://www-sop.inria.fr/oasis/proactive/) and its associated grid component model: Suggested by Jean-Pierre Prost (IBM, France) H: GAT The Grid Application Toolkit: Suggested by Thilo Kielmann I: SAGA: Suggested by Ed Seidel J: PyGridware Python Grid infrastructure: Suggested by Mary Thompson (LBL) K: Broader Coverage in Scheduling and Workflow: Suggested by Lennart Johnsson (Houston) L Commercial Systems: Suggested by Dave Berry (Edinburgh) M: Altair: Suggested by Bill Nitzberg but said he didn’t want to talk! 12

13. Current Workshop Topics • Federated Identity at OGF19 organized by Ken Klingenstein and Satoshi Matsuoka • Semantic Web 2.0 at OGF19 organized by Dave de Roure • 2nd International Workshop on Campus and Community Grids, continuing interoperability at OGF20 organized by Wolfgang Gentzsch, Laura McGinnis, and David Wallom • CommercialWeb 2.0 at OGF21 organized by Charlie Catlett • Possible workshop on Implications of Web2.0 on eScience, OGSA (Grids), Enterprise, Digital Libraries/repositories • Preferred organization of one –day workshops • Invited and Contributed Talks • Panel aimed at summarizing topic as relevant to Grids or Grid technology • After meeting one would • Post presentations • Convert panel discussion into a “review” “synopsis” or “Best Practice” for area covered • Possibly arrange for scholarly publication for follow-up papers 13

14. BP: Best Practice? • There are many grids and experience building and using them. These use “standard” software packages supplemented by more or less “local” work • BP’s, Status reports, Summaries, Reviews presumably capture current understanding of research, practice and experience • These consist of • Integrated reviews and/or • Broad collection of articles such as sets of papers at a focused meeting • Provide a set of links organized by say OGSA categories if possible which link to BKM’s which may or may not be OGF Generated • Portals, Workflow, Science Gateways were covered in OGF workshops • We suggest OGF web pages populated with some existing surveys and solicit additions 14

15. Call forOGF21 Software Provider Forums • A feature of OGF19 was a multi-day track devoted to a Grid Software Providers track. The track was motivated by the OGF GIN Grid Interoperability work and featured the developers of core software that has broad use or interest among Grid system builders. The general idea is recorded below and although the sessions were effective, the audience was not as expected experienced users but rather those interested in a broad survey of the software modules. • For OGF21, we will repeat the experiment but with some changes to encourage a broader audience and greater impact on other important OGF functions. We are interested in Grid Software Providers who wish to present at OGF21 and • a) Have a plausible plan to attract a good audience with preferably “new” OGF attendees. This could be by advertising their session to their existing users. • b) Are willing to participate in interoperability brainstorming sessions with the OGF GIN activity and other OGF Standards and Enterprise representatives. • c) Meet the broad guidelines set in OGF19 Software Provider Forum plans given below.

16. OGF19 Software Provider Forum ExpectationsHow should we change for OGF21? • The audience will typically be those that are building Grids (including software engineers and systems administrators) and aims to present in one place a set of user forums on the software used to build either broad national Grids or those aimed at particular application domains. We expect interchange between providers and users and between the users exchanging experiences among themselves. • We expect the sessions not to be sales pitches but interchanges between providers and users already at least broadly familiar with their product. • Although organized by the OGF eScience function, we expect the track to be important for Enterprise Grids. The software in this track should/could be major consumers of OGF standards and so the sessions could be interesting from this perspective. • We understand that we will only cover some of the critical and innovative Grid software and that we are omitting software largely developed for a single Grid and software from commercial vendors.

17. OGF21 Workshops • At OGF19 we proposed an OGF21 Program: • Day 1: Commercial Web 2.0 • One-day Web2.0 and the Internet Industry Workshop (Charlie) • All invited (Amazon, Google, eBay, Microsoft)? • Day 2 (and perhaps 3): Web 2.0 and Grids • - Implications of Web2.0 on eScience • e.g. MyExperiment • - Implications of Web2.0 on OGSA (Grids) • e.g. Mashups v workflow and REST v SOAP • - Implications of Web2.0 on Enterprise • e.,g. Adoption of Enterprise 2.0 • - Implications of Web2.0 on Digital Libraries/repositories • e.g. Connotea, YouTube …

18. Draft Web 2.0 and Grids Workshop CFP • Collaboration is being revolutionized by the increasing power of communication infrastructure and the Internet which allow both new modes of collaboration and new technologies to support existing approaches. Grids are enabling eScience with scientific collaboratories that will be essential for managing the deluge of information coming from sensors and instruments from the tiniest environmental monitor to distributed high throughput biological devices and the mammoth CERN LHC and shared international satellites. Social or community collaborative networks are being created by intelligent bookmarking tools like del.icio.us and linked back to scientific grids by projects like Connotea. Further Wikis and collaborative collections of MP3 files point to other models of collaborative resource sharing. Web 2.0 encompasses social networking and collaboration sites such as MySpace, Facebook, Flickr and YouTube. There are simple but pervasive compute and storage services from Amazon and online office products from Google. Web 2.0 tends to use mashups, JavaScript, Ajax and REST; Grids use workflow, BPEL, WSDL and SOAP. Further Web 2.0 focuses on usability and enabling all users to create Internet content. • We invite contributions from researchers in academia and industry on the technology practice and user experience on the interface of Web 2.0, Grids, Enterprise systems, Digital Libraries or Repositories and e-Science. • These contributions can be white papers, suggested presentations, panels or tutorials. They can cover applications and/or technology.