Issues and challenges of Agile Development: Some perspectives from Academic Research

1. Issues and challenges of Agile Development: Some perspectives from Academic Research Dr. Sridhar Nerur University of Texas at Arlington

2. Presentation Outline • Are Two Heads Better Than One – Pair Programming Study • Pairs versus Individuals on Design Tasks: A Distributed Cognition Perspective • Case study of XP (Extreme Programming) adoption • Other research projects

3. Are Two Heads Better Than One?A Pair Programming Study (Balijepally, Mahapatra, Nerur & Price, 2009) • Methodology development driven by practitioners • Effectiveness reports were anecdotal or experiential • No systematic effort to provide theoretical explanation of practices • Inconsistent findings on the effectiveness of pair programming • Lack of theoretical and methodological rigor (e.g., Nosek 1998, Williams 2000)

4. Effectiveness of Pair Programming • Research Objectives • To examine the efficacy of pairs vis-à-vis individuals in a programming task • To study the effect of pairing on a programmer’s task satisfaction and confidence in performance • To understand how the above are affected by programming task complexity

5. Theoretical Background • Task Typology • Based on Steiner’s typology (1972) programming tasks are unitary (not divisible among group members), optimizing (quality is more important than speed), and disjunctive (group must arrive at a single solution) • Tasks may be intellective or judgmental ( Laughlin and Ellis 1986) • Programming tasks are relatively intellective with high solution demonstrability

6. Theoretical Background • Nominal Pair (Laughlin et al. 2002) • A nominal pair is created by randomly pairing individuals who worked alone • Comparing the performance of a collaborating pair with the best and the second best members of a nominal pair provides better insight into group performance

7. Theoretical Background • Group Performance • Group members have access to a larger pool of resources (Flor and Hutchins 1991; Hinsz et al. 1997) • Group processes may suffer from process losses due to social loafing (Karau and Williams 1982) • Performance is contingent on task type • With intellective tasks, groups outperform an average individual but rarely perform better than the best individual (Hill 1982, Laughlin et al. 2003) • Quest for assembly bonus effect (when the group outperforms the best individual) still continues (Collins and Guetzkow 1964)

8. Task Complexity H4 Programming Setting Individuals or Pair* Software quality H1 Perceptual Outcomes Satisfaction H2 H3 Confidence in Performance *1 – Best in a nominal pair 2 – Second-best in a nominal pair 3 – Collaborating pair Research Model

9. Research Method and Data Analysis • Lab Experiment • 2 (Individual vs. pair) X 2(Simple Vs. Complex task) factorial design • 120 student subjects (30 pairs and 60 individuals) • Pilot done to check experimental protocol and to determine time needed for the experimental task • 15 minutes for the warm up task and 2 hours for the experimental task • MANCOVA followed by ANCOVA for each dependent measure • Programming ability used as a covariate

10. Results

11. Results

12. Contributions • One of the earliest theoretically grounded empirical research studies on pair programming. • Offers insight into performance of programming pairs • Examines the contingent effect of task complexity on programmer’s performance • Creates a new benchmark to evaluate pair programming performance through the use of nominal pairs

13. Pairs vs Individuals on Design Tasks (Mangalaraj, Nerur, Mahapatra, & Price – under review) • Motivation • Research on pair designing is sparse • Design tasks are cognitively more challenging • Very limited empirical research on design patterns in tasks that are design-centric

14. Theoretical Foundation • According to the theory of distributed cognition, cognition is distributed across problem-solvers (i.e., social interactions), artifacts (i.e., embodied cognition), and time • Two aspects of distributed cognition that are relevant to our research: social and structural (or embodied) • We use pairs (social) and design patterns (cognitive artifacts) in our study

15. Research Objectives • To study the role of design patterns in facilitating the solution of a software design problem • To assess the effectiveness of pairs vs. individuals working on a software design task

16. Theoretical Background • Design problems are ill structured, ambiguous, and have no pre specified solution path (Simon 1973) • Key activities in solving a design problem are • Problem structuring • Searching through the solution space • Problem structuring is facilitated by availability of partial solutions and external representation media (Guindon 1990, Zhang and Norman 1994) • Expert developers use their design experience (stored as design schemas) to constrain the search space (Guindon et al. 1987)

17. Individual/Pair H4, H5a, H5b, H6 Outcome Variables Solution quality Task completion time Task satisfaction With design patterns/without design patterns H1, H2, H3 Research Model

18. Research Method

19. Research Method & Data Analysis • Lab Experiment • 2 (Individual vs. pair) X 2 (Design Pattern Vs. No Design Pattern) factorial design • 92 professionals with no prior pattern experience • Pilot done to check experimental protocol and to determine time needed for the experimental task • Participants were provided a free seminar on patterns • ANOVA/ANCOVA used for analysis • Object-oriented programming experience was used as a covariate

20. Results

21. Contributions • One of the earliest theoretically grounded empirical research on the impact of design patterns on design performance • Demonstrates the benefits of using design patterns • Improves design quality • Reduces task completion time • Enhances developer satisfaction • Consistent with group literature, pairs were found to be better than second-best individuals of nominal pairs

22. Synthesis • Theoretically grounded research provides a deeper understanding of the performance of pairs in software development • Studies demonstrate that pairs outperform second-best individuals of nominal pairs regardless of task type and experience level. • This confirms the long-standing research in group theory, but is counter to the dominant belief in the agile community • The use of nominal groups in this stream of research is a novelty. It permits a more fine-grained analysis that is more meaningful to practitioners

23. Implications • Pairing should be used judiciously • Performance gains are realized when low-performing rather than best developers are paired • The use of design patterns is beneficial • Our studies have established a benchmark for theoretically grounded research in software engineering

24. Case Study – Problems with Adoption of XP practices at ABC Corp.

27. Examples of differences between the two projects

28. Individual Factors • Attitude towards XP • Knowledge of XP • Team Factors • Task management • Team leadership • Technology Factors • Compatibility • Tools support Acceptance of Extreme Programming • Task Factors • Type of application • Project size • Environmental Factors • Budget constraints • Time constraints Figure 3. Extreme Programming Acceptance Framework

29. Other research studies • Efficacy of pairs vs individuals with and without Test-Driven Development • Impact of problem representation on solution quality • Understanding cognitive processes – mental models, cognitive ability • Agile Project Management challenges

30. Questions??