A Sociology of the Grid?

1. A Sociology of the Grid? Ian Foster Computation Institute Argonne National Lab & University of Chicago CarlKesselman Information Sciences Institute, University of Southern California

2. Why We Are Here “With the establishment of large scale multidisciplinary production Grid infrastructures such as the EGEE, OSG, DEISA, TeraGrid, or NAREGI, the concept of Virtual Organizations (VO) has been constantly refined and efficient management of VOs and their policies is becoming one of the central topics for these infrastructures.”

3. “The Anatomy of the Grid,” 2001 The … problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource -brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization (VO).

4. Examples “The application service providers, storage service providers, cycle providers, and consultants engaged by a car manufacturer to perform scenario evaluation during planning for a new factory” “Members of an industrial consortium bidding on a new aircraft” “A crisis management team and the databases and simulation systems that they use to plan a response to an emergency situation” “Members of a large, international, multiyear high-energy physics collaboration”

5. From the Organizational Behavior and Management Community “[A] group of people who interact through interdependent tasks guided by common purpose [that] works across space, time, and organizational boundaries with links strengthened by webs of communication technologies” — Lipnack & Stamps, 1997 • Yes—but adding cyber-infrastructure: • People  computational agents & services • Communication technologies  IT infrastructure Collaboration based on rich data & computing capabilities

6. NSF Workshops on Building Effective Virtual Organizations [Search “BEVO 2008”]

7. Two Perspectives • Technology used to enhance collaboration (Computer Supported Collaborative Work) • Collaboration used to enhance technology

8. What is an Organization? • A organization has an identity and a purpose, which it seeks to fulfill within its environment • The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services • The organization’s performance can be evaluated with respect to various metrics Is a virtual organization any different?

10. “Where am I going to eat tonight?” “I can’t solve this problem alone—I need to involve my buddies Erwin and Miron” It looks like you’re creating a VO Get help withcreating the VO Just create the VO without help From: Ian To: Erwin, Miron Subject: Help me find a restaurant

12. Building a Virtual Organization • Determine policy • Negotiation, trust management • Determine membership and roles • Terms of engagement • Virtualization & integration of providers • Create VO-wide services • Global behaviors • Manage work • Collaborative problem solving, workflow management • Manage the VO • Monitor performance, report metrics

13. A B 1 1 10 10 1 A B 1 2 1 2 16 Defining Community: Membership and Laws • Identify VO participants and roles • And map participants to attributes and roles • Specify and control actions of members • Empower members  delegation • Enforce restrictions  federate policy

14. Security Services Objectives • It’s all about “policy” • Define the VO’s operating rules • Security services facilitate the enforcement • Policy facilitates “business objectives” • Related to goals/purpose of the VO • Security policy often delicate balance • Legislation may mandate minimum security • More security  Higher costs • Less security  Higher exposure to loss • Risk versus rewards

15. Policy Challenges in VOs • Restrict VO operations based on characteristics of requestor • VO dynamics create challenges • Intra-VO • VO-specific roles • Mechanisms to specify/enforce policy at VO level • Inter-VO • Entities/roles in one VO not necessarily defined in another VO

16. Core Security Mechanisms • Attribute Assertions • C asserts that S has attribute A with value V • Authentication and digital signature • Allows signer to assert attributes • Delegation • C asserts that S can perform O on behalf of C. • Attribute mapping • {A1, A2… An}vo1  {A’1, A’2… A’m}vo2 • Policy • Entity with attributes A asserted by C may perform operation O on resource R

17. Trust in VOs • Do I “believe” an attribute assertion • Used to evaluate cost vs. benefit of performing an operation • E.g., perform untrusted operation with extra auditing • Look at attributes of assertion signer • Rooting trust • Externally recognized source, e.g., CA • Dynamically via VO structure  delegation • Dynamically via alternative sources, e.g., reputation

18. Building Blocks User A is an admin • Attribute Authority (ATA): • Issue signed attribute assertions (including identity, delegation, & mapping) • Authorization Authority (AZA) • Makes decisions based on assertions & policy User B is a member ATA User B can use service X

19. VO Policy at a Service VO ATA ATA: Attribute Authority AZA: Authorization Authority Resource AZA Resource ATA WS Resource WS-Subject GT4 authorization and delegation services provide first implementations

20. VO ATA VO AZA Establishing VO-Wide Policy ATA: Attribute Authority AZA: Authorization Authority Subject ATA Resource AZA Subject AZA Resource ATA WS Resource WS-Subject GT4 authorization and delegation services provide first implementations

21. Attribute Mapping VO 1 VOUser A Delegation Assertion User B can use Service X Resource Admin Attribute VO AZA VO ATA VO-1 Attr  VO-2 Attr Mapping ATA VO Member Attribute VOUser B VO Member Attribute Service X VO 2 Service

22. Protected Health Information Problem • What do we want? • Use clinical data for research • Share clinical data, make research data available • Reuse same infrastructure • Image exchange between health providers • Patient authorizes use of data – consent process • Intact unmodified DICOM workflow for diagnostics • De-identified DICOM workflow for research (Modality profiles) • Group authorization problem: Patient data–to-user (Physician/Researcher) relationship not manageable!

24. Patient Authorized Grid Image Workflow MEDICUS (Erberich et al.)

25. HIPAA Compliant Research Access MEDICUS (Erberich et al.)

26. PHI safe workflow with PCA

27. VO as a Service (VOaaS) • Virtual organizations integrate participants and resource providers • Participants are selected or self assemble • Select “best of breed” providers for VO services • Much of this process can be automated • Provisioning of enabling services, at least Function Resource

28. VOs Assemble Services • Integrate services from various sources • Virtualize external services as VO services • Deploy new services for the VO Community Content 3 2 Services Provider Services 5 1 Capacity Provider 4 Capacity

29. VOs Assemble Services Domain-dependent Domain-independent Simulation code Expt design Simulation code Content Expt output Certificate authority Electronic notebook Tele-op monitor Simulation server Services Portal server Data archive Metadata catalog Capacity Servers, storage, networks Experimental apparatus

30. Providing VO Services • Integrate existing services • Delegate and deploy capabilities/services • Provision service to deliver defined capability • Configure execution environment • Host higher-level functions • GRAM, Workspace Service, EC2, … • Coordinate and compose • Build new functions from individual services

31. Application Workflow Application Interface Embedded VO management Mgmt Interface WS-Agreement DAGMan Capability provisioned for VO Mgmt Interface WS-Agreement Mgr Glidein GRAM Managed Service Mgmt Interface WS-Agreement Cluster Virtualization and VOs: Its Turtles all the Way Down Provisioning, management and monitoring at all levels

32. grid Remote Application Virtualization Infrastructure Builds on Introduce Define service Create skeleton Discover types Add operations Configure security Wrap arbitrary executables Appln Service Create Store Advertize Discover Transfer GAR Invoke; get results Deploy Service Authoring and Deployment RaviMadduri Introduce Repository Service Index service Container gRAVI: Ravi Madduri et al., Argonne/U.Chicago & OSU

33. Registries(E.g., caBIG)

34. VM Factory create new VM image request VM EPR Create VM image VM Repository inspect and manage use existing VM image Client Resource VM Manager VM deploy & suspend start program Workspace Service(For When You Want a Virtual Machine) Kate Keahey TimFreeman

35. Composing Services

36. Service Composition:Data Replication Service Pull “missing” files to a storage system Data Location Data Movement GridFTP Local ReplicaCatalog Replica LocationIndex Reliable File Transfer Service GridFTP Local Replica Catalog Replica LocationIndex Data Replication List of required Files Data Replication Service AnnChervenak “Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005

37. “Provide access to data D at S1, S2, S3 with performance P” S1 S2 D ServiceProvider S3 Replica catalog, User-level multicast, … “Provide storage with performance P1, network with P2, …” S1 D S2 ResourceProvider S3 Decomposition EnablesSeparation of Concerns & Roles S1 User S2 D S3

38. Policy, Revisited • Traditionally policy is enforced at end points, integrated with application • E.g., PDP call-out in GT container • We can also apply policy at the VO level • Define interactions between services at the organizational level • Factor policy out of service implementations

39. Policy-Driven Service Oriented Architecture • Need stand-alone policy engine to coordinate at VO level • Connection between application policy and infrastructure policy (dynamic provisioning) • Policy extension points designed into services allow • Coordination at VO level • Dynamic policy enforcement across services and service oriented infrastructure Web Services 2.0: Policy-driven Service Oriented Architectures Thomas B Winans and John Seely Brown

40. A Traditional View ofthe “Grid Problem” Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations Too limited a view

41. We Need an End-to-End Perspective • A organization has an identity and a purpose, which it seeks to fulfill within its environment • The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services • The organization’s performance can be evaluated with respect to various metrics Then focus on clear identification of roles, separation of concerns, isolation of policy