Petter Øgland ECIS-17 Conference, Verona, June 10, 2009

1. Measurement, feedback and empowerment: Critical systems theory as a basis for software process improvement Petter Øgland ECIS-17 Conference, Verona, June 10, 2009

2. Problem • How to select a design strategy for software process improvement (SPI) for achieving optimal payoff in an organization that is not seriously motivated to do SPI?

3. Two perspectives • Brunsson et al (2000): Organizations want to be seen as complying with ISO 9000 and other SPI standards, but they do not want to do what is required • Flood (1993): Critical social theory (Adorno, Habermas, Foucault, …) should be used as a foundation for TQM (e.g. SPI) to liberate and improve social standards while improving business processes

4. Hypothesis • If we look at SPI as knowledge management, then the situation becomes political (relationship: knowledge/power), we can use Flood’s idea, and the SPI implementation strategy becomes optimal

5. Total Systems Intervention (Flood & Jackson, 1991) • Creativity • Use Morgan’s metaphors for describing the organization (problem) • Choice (SOSM) • Viable Systems Methodology (Beer, 1972) • Soft Systems Methodology (Checkland, 1981) • Critical Systems Heuristics (Ulrich, 1983) • … • Implementation

6. CST PSM OR 3rd loop 2nd loop 1st loop Why? What? How? Triple loop learning (Flood & Romm, 1996):2-loop learning (Argyris, 1978) + “might is right”?

7. Real world problem Engineering design Documented specification of solution Formulate Solve Theory Model Decision Implementation Documented evaluation of solution New knowledge Real world solution Monitoring Design of experiment for testing hypotheses:Design science = design QMS & evaluate Theory

8. Case study: • Unit within public sector organization • Approx 20 people (system designers & computer programmers) • Average age = 40, male/female = balanced • Working according to life cycle model • There is a documented QMS • First version of information system established 1998; system is now mature and work is concerned with annual updates and new functionality • Generally seen as one of the better units of the organization (“role model”)

9. Case study: Technical results 0% = Bad Unstructured changes in format (”improvements”) register as decline in documentation quality

10. Evolution of QMS: social perspective • Year 1: Distribution of SPI results sideways and upwards. Emotional stir and frustration. Complaints to head of corporation (saved by SPI owner) • Year 2: Small improvements, people complain that “products are important, not processes” • Year 3: Audits show that not only process is of low quality, but predictions about product development are bad too. • Post-experiment: The methodology is rewritten to achieve better scores without achieving better quality

11. The effect of TSI on SPI µ¹ µ² TSI H0: µ¹ = µ² Not rejected at significance level 0.022 using t-test with four degrees of freedom

12. Discussion: A suggested TSI strategy for playing the SPI game Create vertical tension by reporting SPI results one level above internal customer Create horizontal tension by benchmarking SPI results Make sure the QMS owner is the winner of the political game of SPI

13. Conclusion • SPI standards can imprison organizations in “fake quality” (false beliefs) • TSI suggests a path toward “real quality”, through critical systems theory (CST), but depends on organizational willingness to admit to problems and commit to methods • This study presents a different way of implementing TSI, designing SPI as a “conflict machine”, minimizing the needs for admitting and committing • Three years of data was not sufficient for statistical reasoning, but the phenomenological aspects of the study showed that the TSI-based SPI strategy was successful in the context of the case