Debugging Integrated Systems: An Ethnographic Study of Debugging Practice

1. Debugging Integrated Systems: An Ethnographic Study of Debugging Practice Thomas Østerlie & Alf Inge Wang Debugging Integrated Systems

2. Overview • Background • Debugging • Motivation • Sensemaking • Research setting • Overview Gentoo • Gentoo’s formal debugging process • Five characteristics of debugging practice in Gentoo • C1: Spans a variety of operating environments • C2: Collective • C3: Social • C4: Heterogeneous • C5: Ongoing • Concluding remarks • Brief summary • Transferability of findings • Implications

3. Debugging: The traditional view • Trace a causal chain from reported failure to its corresponding fault • Access to and control over source code critical for this form of debugging

4. Motivation • Increased focus on systems integration • Component-based development • Information systems and enterprise integration • Service-Oriented Architecture • Debugging integrated systems • Software being integrated is developed and maintained by third-parties • Integrators have to debug systems without access to the source code of the components being integrated • What do systems integrators do in practice when debugging? • Little is known about the debugging of integrated systems • Need to understand what is going on before improving on the process

5. Sensemaking • Problem solving • Individual cognitive activity • Clearly defined problems and resources to solve the problem • Problem: software failure • Resources: tools and techniques for locating the fault • Problem setting (Schön 1991) • Problems do not present themselves as givens • Make a situation that is puzzling, troubling, uncertain into something that makes sense • Constructing problems from the materials of problematic situations • Sensemaking (Weick 1995) • ‘What is going on?’ • A group’s collective experiences of a problems situation progressively clarified • Action precedes understanding • Active engagement with the problem situation • Understanding is retrospective

6. Research setting: Gentoo • Community of volunteer software developers • 320 official Gentoo developers • Distributed across 38 countries, 17 time zones • Develop and maintain a software system for integrating third-party OSS packages with various Unix operating systems • The Gentoo software distribution – Integrator’s view • 8,480 third-party packages supported • One installation script for every version of each supported package • Total of 23,900 installation scripts in package database • Total SLOC of 671,971 in package database • Supports 6 different Unix operating systems (5 processors on GNU/Linux) • Gentoo installation – User’s view • Runs on a local computer • Local copy of the Gentoo package database • Uses the Gentoo package manager to integrate third-party OSS packages with the local system

7. Formal process of debugging in Gentoo

8. C1: Spans a variety of operating environments • Variety of operating environments among individual Gentoo installations: • Configuration of individual packages • Optionals • Virtuals • Operating system • System evolution • “Variation kills reproducibility” • State of individual Gentoo installations often impossible to replicate • Reproduction of reported failure therefore often impossible • Problem situation, rather than clearly defined software failures

9. C2: Collective • User and developers work together to make sense of the problem situation • Collaborative debugging • User provides data - creates the materials of the problem situation • Developers interpret data, request new data • Collaborative environment • User to developer communication: Problem reports • Developer to developer communication: IRC channel

10. C3: Social • Workload • Priorities • Reproducible vs. irreproducible • Curb work load vs. retaining users’ interest • Responsibilities • Formal roles • Actual problem • Negotiate to bridge reality of formal roles with reality of actual problem • Determining responsibility inherent in sensemaking process

11. Exhibit: Enacting responsibilities

12. Summary of findings • Debugging integrated systems is a collective sensemaking process • Cyclic rather than linear • Influenced by technical as well as social factors • More than individual problem solving • Process driven by plausibility rather than accuracy

13. Transferability of findings • Presented findings to systems integrators • International network of researchers and practitioners • R&D department of a global telecom • National consultancy specializing in systems integration • Similarities between commercial and volunteer context • Finding the problem is the problem • The role of organizational issues • The precarious relationship between systems integrator as provider and customers • Similarities between systems integration and application development (tentative) • Finding the problem is the problem • The role of organization issues

14. Implications • For research #1: Distinction corrective maintenance and debugging • Not useful when studying practice • The practice of debugging closely intertwined with the administration of the corrective maintenance process • For research #2: For systems integrators the software failure is not an unproblematic phenomenon • Subject to interpretation and negotiation • What constitutes a software failure contingent upon the situation (workload, priorities, responsibilities, access to technical data) • Clear relation between error in code and observed failures too simplistic • Need better models for understanding software failures in practice • Implications for practice • Difficult to determine what data is relevant prior to engaging with the problem • Comprehensive classification schemas of limited use for users reporting problems • Defect tracking systems need to support interaction between the reporting user and the system integrators debugging the problem

15. Thank you for your time.Questions?