Problems and Software Process Improvement

1. Problems and Software Process Improvement Jim Kelly Zenkara Pty Ltd jim@zenkara.com

2. Outline • Present an overview of the research • Describe survey conducted • Describe results and feedback • Provide some analysis; and • Generate some discussion

3. Overview of the research 1 • What we’re looking at, and what we’re not. • In our scope we have reviewed the problems that CMMI* (or an alternative SPI model) addresses • We have not included the actual problems experienced in deploying SPI initiatives (though this is related) * Where CMMI is referred to, you can generally substitute alternate SPI models/methods

4. Overview of the research 2 • Examining the problems addressed by using Software Process Improvement models/methods • Hypothesis is that some organisations do not identify the specific goals that CMMI is intended to address • If organisations better defined the specific goals that CMMI is to address, there would be more successful deployments

5. Overview of the research 3 • Prime focus is on SW CMM and CMMI for software organisations (although other models are being examined) http://www.software.org/quagmire/

6. Details of survey 1 • Conducted interviews with 12 Software Process Improvement “experts” who operate in Australia either embedded in software organisations or externally who provide advice to software organisations. • Considered numerous models/methods: • SW CMM, CMMI, ISO9001, SPICE • Results were qualitative and descriptive

7. Details of survey 2 • Conducted face to face and phone interviews (45m to 1.5hrs) with an indicative line of questions: • How long have you been active in CMMI? • What training have you had in CMMI? • What is your rationale for using the CMMI? • What benefits of the CMMI were identified to contribute to your organisation? • What tools were used to identify the problems? • How do you know they were appropriate? • “CMMI” was used for the majority of interviews, but was broadened for some organisations to SPI.

8. Results and Feedback 1 • For large organisations (>100 staff), PI model is mandated by corporate policy or by customers. • Benefits of the CMM/CMMI and other models are often not identified • Costs of PI model deployment often not exhaustive - e.g. only recording PI staff effort, and not operational involvement

9. Results and Feedback 2 • Organisations which use SW CMM or CMMI generally also use at least one other SPI model/method (e.g. ISO9001, EFQM, SPICE, Personal Software Process, Team Software Process) • Records of initial justification for initiating Process Improvement initiatives often not kept

10. Results and Feedback 3 • Some organisations have “links” from their Process Improvement efforts to their organisation’s Business Goals/Objectives • However the links are mostly tenuous with little details about what the PI effort will do to e.g. improve productivity, schedule, budget, product quality.

11. Results and Feedback 4 • No real structured approach to finding and understanding problems to be solved. • Most experts interviewed had an unstructured view of organisational problems – very much a mess* * Ackoff’s definition of mess: world.std.com/~lo/96.02/0009.html

12. Results and Feedback 5 • Many experts not immediately aware of top 5 problems with their organisation • Many also did not have knowledge of specific problems which have been solved over the past 1-2 years

13. Results and Feedback 6 • Institutionalisation is often seen as most important problem (particularly in mature organisations!?) • CMMI experts who are in an immature organisation are aware of the problems and are aware that the problems are not being addressed in a structured approach, but do not use formal techniques (Pareto, SPC).

14. Initial analysis 1 • Rationale for selecting specific SPI models is far from clear in many organisations. • “Links” with business goals must be made explicit: how exactly will the PI contribute? • Simply saying ‘improve productivity’ or ‘reduce rework’ is not adequate – e.g. what component or process of the CMMI precisely is going to help?

15. Initial analysis 2 • clearly identify benefits • How they were calculated • Which specific CMMI actions helped and how • Use the impact of the CMMI to help identify current problems and areas for improvement

16. Initial analysis 3 • Look at the current effort in multi-SPI-model initiatives: • Interactions • Efficiencies • Quite often improving something which satisfies requirements of one model/method will also satisfy part of another.

17. Initial analysis 4 • More mature organisations (CMMI L3+) often only see problems within the scope of the CMMI

18. Useful Websites • www.sei.cmu.edu • www2.umassd.edu/SWPI/1docs/SPResearch.html • www.sqi.gu.edu.au • www.psmsc.com • www.swquality.com/users/pustaver/index.shtml • nicta.com.au/uploads/documents/Why%20Organisations_%20Dont_%20Use%20CMMI.pdf • www.dtic.mil/ndia/2004cmmi/CMMIT7Thur/CMMIUsersGroupjmsiviy.pdf • www.stsc.hill.af.mil/crosstalk/2002/02/menezes.html

19. Demonstration • Break into groups of two – preferably with someone you don’t know • Introduce yourself • Individually read and answer the red sheet • Review your team member’s problems • Try to come to a decision about which problem your team would identify as #1 priority

20. Q&A • Does this correlate with your experiences? • What is missing? • What do you interpret differently? • Which models do YOU use?

21. Who are we? Zenkara provides advanced technology for mission-critical software engineering companies. Head Office: 2/27 Waverley Street Annerley Qld 4064 Australia PO Box 1622 Milton QLD 4064 Mobile:+61 (0)410 616125 | direct: +61 (0) 7 3892 6771 email: jim@zenkara.com web: http://www.zenkara.com ABN: 85 050 409 660