The Architecture of Coordination

1. The Architecture of Coordination James Herbsleb Carnegie Mellon University jdh@cs.cmu.edu http://conway.isri.cmu.edu/~jdh/ The author gratefully acknowledge support by the National Science Foundation under Grants IIS-11 0414698, IIS-0534656, and IGERT 9972762

2. Architectures and Organizations • Level of an industry • E.g., modular clusters • Level of an organization • E.g., matching hypothesis • Level of a practicing architect, manager, or engineer • Architectural decisions define coordination problems • Understanding and managing impact of decisions

3. Today . . . • How can you measure the match of an organization with the structure of the software it is producing? • Coordination theory about the “modularity mismatches” – the failures or limits of modularity

4. Product Structure: Conway’s Law • “Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.” M.E. Conway, “How Do Committees Invent?” Datamation, Vol. 14, No. 4, Apr. 1968, pp. 28–31.

5. Teams Components Homomorphism Organization Software Conway’s Law

6. Teams Components Organization Software Designing Coordination Problems

7. Teams Components Organization Software Designing Coordination Problems ? What kind of coordination is required?

8. Socio-Technical Congruence and Productivity cr11 … cr1n crn1 … crnn a11 … a1n ak1 … akn d11 … d1k dk1 … dkk a11 … a1k an1 … ank Measuring Coordination Requirements (CR) Concept Task Assignments Task Dependencies Coordination Requirements (A) (D) (AT) (CR) = X X

9. Coordination in a Distributed Project • Software firm, 1 product, 114 developers • 8 teams, 3 locations, all in US • Product separated into components • Each component assigned to 1 team Cataldo, M., Wagstrom, P., Herbsleb, J.D., Carley, K. (2006). Identification of coordination requirements: Implications for the design of collaboration and awareness tools. In Proceedings, ACM Conference on Computer-Supported Cooperative Work, Banff Canada, pp. 353-362.

10. A Word About Tools and Data • Use archival data from large software development project • Modification request (MR) system • Users, testers, developers request changes • Bug fixes, new functionality • Version control system • Maintains all changes to all files • Some set of changes correspond to each MR • Has data about who made change when • Communication data • IRC chat • Discussions within MR system

11. Socio-Technical Congruence and Productivity cr11 … cr1n crn1 … crnn a11 … a1n ak1 … akn a11 … a1k an1 … ank d11 … d1k dk1 … dkk Measuring Coordination Requirements (CR) Concept Task Assignments Task Dependencies Coordination Requirements (A) (D) (AT) (CR) = X X Developer modified files Files changed together Transpose of developer modified files Who needs to coordinate with whom Data

12. Volatility in Coordination Requirements Members of other teams Change in coordination group Proportion Week

13. Socio-Technical Congruence and Productivity cr11 … cr1n crn1 … crnn ca11 … ca1n can1 … cann Measuring Congruence Coordination Requirements (CR) Actual Coordination (CA) • Team structure • Geographic location • Use of chat • On-line discussion Diff (CR, CA) = card { diffij | crij > 0 & caij > 0 } Congruence(CR, CA) = Diff (CR, CA) / |CR|

14. Predicting Resolution Time (* p < 0.05, ** p < 0.01)

15. Average Level of Congruence for Top 18 Contributors

16. Average Level of Congruence for the Other 94 Developers

17. The Story So Far . . . • Coordination requirements are highly volatile. • Congruent coordination activities are associated with performing the work faster. • The top developers behave much more congruently than the rest. • Have replicated congruence finding on GNOME projects • Have experimented with many different ways of computing dependencies • Logical dependencies the most predictive by far

18. Theory • Build on modularity/architecture literature • Product structure and task structure • Build on network analysis • Network attributes matter • Relations among people, decisions, components • Methodology: build on idea of logical dependencies

19. Coordination: Five Propositions • P1:Design progresses by making decisions. • P2: Decisions are linked by constraints in a potentially large and complex network. • The “decision network” • P3: The need for coordination among individuals arises from • Properties of the decision network • Assignment of decisions to people • P4: Effective coordination is the result of coordination actions, moderated by coordination capacity. • P5: Coordination breakdowns occur when effective coordination is insufficient for coordination needs

20. Example Domain: Field Robotics

21. Example Network: Field Robotics

22. Coding Method • Meeting segments Avionics Optics Software

23. Coding Method • Meeting segments • Divide into decision discussions Avionics Optics Software

24. Coding Method • Meeting segments • Divide into decision discussions • Code according to components involved in the decision discussion • Co-occurrence much like logical dependency Avionics Optics Software Mobility Camera

25. Example Network: Field Robotics

26. Hardware Discussions Newman Modularity: .39

27. Software Discussions Newman Modularity: .03

28. Network Theory of Coordination Decision Network Task Assignment Coordination Requirements Coordination Effectiveness Outcomes Coordination Actions Coordination Capacity

29. Challenges • Planning • Predicting coordination requirements • Assessing coordination capacity • Choosing optimal coordination actions • Coordinating • Monitoring for unexpected mismatches • Organizational tactics, architectural tactics • Theorizing • Principles, patterns for network dependencies • Prior theories as special cases