Avgousta Kyriakidou & Will Venters Information Systems and Innovation Group

1. Distributed large-scale systems development: Exploring the collaborative development of the particle physics Grid Avgousta Kyriakidou & Will Venters Information Systems and Innovation Group Department of Management The London School of Economics This research was undertaken as part of Pegasus EPSRC: Grant No: EP/D049954/1 www.pegasus.lse.ac.uk

2. Overview • How large-scale systems are developed in a distributed way through a case study of a Grid development project – the LCG (Large hadron collider (LHC) Computing Grid) . • Explore the collaborative practices employed by particle physicists, focus on collaboration, history & time. Provide practical recommendations to those involved in globally distributed systems development. • Grid: Computers, storage devices and sensors connected by very fast networks – and Middleware to “virtualise” these resources. Grid computing for the LHC “demands” distributed systems development practices. • Activity as unit of analysis (minimal meaningful context for individual actions).

3. Global Systems Development • Technological advances allow complex software development projects to run across geographically distributed environments (Oshri, Kotlarsky et al 2008). • GSD Complicates communication, coordination and control (Damian and Moitra2006). • Limitations of traditional systems development and agile practices not the silver bullet to GSD.  GSD demands new development practices since the nature of the problem and the environment are now different. • How to ensure Agility of GSD practices? • What is the impact of the shift from traditional IT systems to information infrastructure such as Grids?

4. Theoretical Framework • Activity Theory: how technology is collaboratively constructed to fulfil the objectives of a global community (Nardi 1996) • Present/frame the LCG project as a complex activity system influenced by the context, the community’s rules, norms and collaborative way of working. • Understand the Grid’s development as a series of contradictions between the elements of this activity system. • Activity Theory is driven by contradictions.:“systematic contradictions, manifested in disturbances and mundane innovations, offer possibilities for expansive developmental transformations” (Engestrom 2000). • Conceptualisation of collaborative breakdowns and tensions.

5. Activity Theory Tools Unit of analysis – entire human activity in the collaborative context (Guy 2003). Activity is the context Transformation Process Desired Outcomes Subjects Object Rules Community Division of labour (Engestrom 1987)

6. CASE: LHC @ CERN • Demands globally distributed Grid of around 100,000 CPU(equiv). • Distributed due to funding and expertise availability. • A significant systems development challenge demanding distributed collaborative effort (Berman et al 2003). • PP well-known for its distributed nature (Chompalov et al 2002) • LCG cannot used plan based approach • Virtual meetings for development. Video-conf., emails, blogs, wikis and travel.

7. Developers’ Activity System • Desired Outcome: They are waiting for the LHC to produce data to analyse. • Object: Development of the LCG was realised when they understood that a Grid was necessary. • Formed global virtual alliances: “What physicists want to do cannot be done by a small group, its needs a large collaboration.” • Subjects: Global community of particle physicists and computer scientists.

8. Developers’ Activity System • Tools: Activity is driven by the LHC. • Plan-based approach considered inappropriate for LCG. • No full overview of the system – modularised, fluid architecture • Draw on down-to-earth approaches embedded in PP tradition and history. • Pragmatic and improvisational in response to tight deadlines: “[PP] are more pragmatic in computing”. “they do [computing] just to resolve a problem for now” • Reactionary, flexible, adaptable. • “how can you possibly do a formal software engineering approach… if we are going to change it?” • Natural selection: hacked solutions.

9. Developers’ Activity System • Community: • “Elite among sciences and among physics” (Traweek 1988). • Communitarian (Knorr-cetina 1999). • “we run big collaborations across every nation on earth and they always work” • “Set up to look like an experiment” • Shared goals and “sacred causes”. • Belief in “genius” nature of work – individual skill, competence and creativity. • Trust as central element.

10. Developers’ Activity System • Rules: Collaboration is seen as inherent. Developing a strong sense of community. • Socialisation and Communication. • Improvising • Skills development via word of mouth.

11. Developers Activity System • Division of Labour: • Volunteerism and flexible • “we don’t regard our roles as having fixed boundaries.” • “No strict hierarchy” • Politics is dispersed • “why was the web invented here? Because Tim had the freedom from hierarchy to spend a bit of time investigating something of interest to him.”

12. Activity Theory Tools: SD Practices, Methodologies, Programming language Subjects: PP and Computer Scientists Object: Grid for LHC Outcomes: Support for LHC Transformation: Collaborative SD Rules: Collaboration, budgets Division of labour: Fuzzy, Voluntary, lacking leadership Community: Collaboration, trust, pragmatism

13. Inner Contradictions… From the AT framework we observe contradictions: • Between physicists and computer scientists • Between developers and users • Between methodology and practice. • With the extreme size of the LHC experiments and previous experiments • Contradictions are arguably resolved through “expansive learning” – new technology, tools & work practices…

14. New practices… • “Clusters of competence” • Balancing experimentation with discipline • A sense of belonging

15. Conclusions • Lessons: • Structure the development effort in clusters of competence, • Encourage temporary co-location of developers, • Combine flexibility/agility with structure/discipline, • Create a sense of belonging and therefore construct identity for those involved in the development, • Facilitate human communication both through virtual means and face-to-face (at least every couple of months), • Create a trustworthy environment, • Have clear shared goals and rationale.

16. Distributed large-scale systems development: Exploring the collaborative development of the particle physics Grid Avgousta Kyriakidou & Will Venters Information Systems and Innovation Group Department of Management The London School of Economics This research was undertaken as part of Pegasus EPSRC: Grant No: EP/D049954/1 www.pegasus.lse.ac.uk

17. Why Activity theory? (Korpela, Mursu et al. 2002): AT can provide a theoretically founded but detailed and practicable procedure for studying systems development as a real-life collaborative work activity in context. (Kuutti 1996): AT is a powerful and clarifying descriptive tool, rather than a descriptive theory focusing on understanding and studying different forms of human activity and work practices as developmental processes with both individual and social level interlinked. (Crawford and Hasan 2006): provides a useful paradigm for the ways in which human experience, needs and creativity shape the development and effectiveness of emerging technologies. (Leontiev (1978): Collectiveness and the notion of different actors sharing the same goals and constructing the same meanings are at the core of AT

18. Why Activity theory (cont..) (Lehenkari and Hyysalo 2003): AT is particularly suitable to be used in studying contextually embedded interactions, as it contains features such as the recognition of actors, mediations, historicity, constructivity, dynamics etc.  helps the researcher to understand how these contextually embedded interactions influence the development and the systems development practices employed when developing the Grid (Engeström 1987):AT does not offer ready-made techniques and procedures for research, rather its conceptual tools must be concretized according to the specific nature of the object under study Offersa place for innovation and creativity (Mwanza 2001): Activity notations - all possible interactions between the elements of the activity system. Questions that are specific to a particular combination can be then generated, and together with the notion of contradictions can guide the researcher’s field work and interpretation of findings.