SPI – past and future

1. SPI – past and future Tor Stålhane IDU / NTNU

2. What has happened – 1 Most of the SPI that was done before 1980 – 1990 was based on • the work of Deming on TQM (Total Quality Management) • ideas from DoE (Design of Experiments), with a scientific basis using statistical analysis. GQM from the University of Carnegie Mellon in cooperation with NASA, had a large impact. GQM focuses on: • Data collection based on scientific principles • Data analysis based on statistical methods

3. What has happened – 2 The need for sound scientific results was driven by the ambition of arriving at general results. This led to the use of statistical analyses with a high level of significance – e.g. 95% – which again led to the need for a large amount of data, which • was difficult for many small organizations. • led to the need for collecting data over a long time if the organization did not have a large project. • led to a long lead time, which made the improvement process slow. As long as the introduction of new tools, methods and processes was slow, this was not a serious problem.

4. The late 90’s – 1 The main problems were the • amount of data needed in order to use statistical analysis • calendar time needed to collect the data The SPI community changed to qualitative methods • not founded in traditional science • founded in business and social sciences, e.g. force field diagrams, affinity diagrams and root-cause analysis.

5. The late 90’s – 2 The qualitative methods from business and social sciences are • simple • rely mostly on subjective data picked from the participants’ knowledge and memory • consume a considerable amount of resources. A post mortem analyses in a five person project will • consume a person-week to collect and analyse the data. • not allow you to identify any effect of an improvement until the end of the next project.  

6. Agile software developmentand SPI

7. What’s new – 1? SPI will by its very nature always contain the activities: • Identify things that we want to improve • Decide how to improve them • Implement the necessary changes What varies is: • When we do improvement • How we identify what we want to improve • How we make decisions

8. What’s new – 2? The main thing with agile SPI is the opportunity offered by splitting the project into several short iterations. Traditional software development: • Run project • Discover problems • Suggest and implement improvements • Try out in the next project Agile software development • Run an iteration • Discover problems • Suggest and implement improvements • Try out in next iteration

9. Agile SPI – 1 The current problems made ideas from agile development appealing. A partly agile approach called Rapid Process Improvement (RPI) was suggested by I. Seward et al. and presented at an SPI conference in Gothenburg in 2000. The Scandinavian firm Tieto-Enator has developed a process improvement method with focus on checklists and short iterations.

10. Agile SPI – 2 The current focus is not on collecting and analysing large amounts of data. We have two competing concepts. • Data-heavy concepts such as • QIP (Quality Improvement Paradigm) which uses GQM (Goal Question Metrics) to collect and analyse large amounts of data • TQM (Total Quality Management) • Scrum's sprint retrospectives. In between these two extremes there is plenty of room for concepts like checklist driven SPI, post mortem analysis and other, related ideas.

11. Agile SPI – 3 The principles of agile development request that we: • at regular intervals reflects over how to become more effective • tunes and adjusts our behaviour The focus is on • the need for SPI within the on-going project • moving process control from the organization level to the practitioners • the immediate use of experiences of the developers in improving the on-going project

12. Agile SPI – 4 Expected benefits are e.g.: • Better collaboration between SPI project and developers • Better able to adopt changes • Quicker deployment of new process The emphasis is on face-to-face communication as the primary means for conveying information • To the development team • Within the development team

13. Agility and communication The most important reason why agile development has been a success is that it improves communication. For software development – and for many other areas – bad communication is the root of all evil.

14. Communication efficiency – 1

15. Communication efficiency – 2

16. Communication efficiency – 3 • Physical proximity. The closer people are to one another the greater the opportunities for communication. At one end of the spectrum two people can be working side-by-side and at the other end of the spectrum two people can be in different buildings. • Temporal proximity. Whether or not two people are working at the same time affects communication. • Amicability. Cockburn believes that amicability is an important success factor. Greater amicability => • greater amount and quality of information will be communicated • less will be concealed.

17. Communication efficiency – 4 Nothing beats face-to-face communication – preferably supported by a white board. Nothing is worse that communication through large amounts of detailed documents For many companies, however, the preferred way of communicating with customers and with assessors is through detailed documents, which is the least effective way of communicating.

18. Challenges Many agile methodologies suggest iterative SPI as an important part of development but lack: • Specific, clear procedures for this activity • Focus and long duration of perspective • Means for validating the implemented process improvement • Systematic follow-up of improvement activities

19. SPI and Scrum – 1 Scrum has an explicit SPI activity at the end of each sprint. Most of the material on Scrum SPI is found in blogs. Some important points from one of the bloggers: • Establish trust so that everybody feel free to speak his mind. • Identify the facts – data collection, one way or another. • Identify the team's strengths – what do we do well • Collect ideas for improvement – where can we improve ourselves and how • Consolidate and prioritize the ideas.

20. SPI and Scrum – 2 Small datasets in agile SPI may result in improvements that are not as good as the ones achieved using e.g. GQM. The agile paradigm’s strong points: • No preparations – e.g., courses – are needed. • The time span we consider in the retrospective – one sprint – is usually short and the participants remember all important facts and events – no data collection is needed during development. • The effect of any SPI action that we implement can be observed in the next sprint. • We do these SPI activities for this project – quick pay-back.

21. Benefits of agile SPI According to Ericsson, the befits of agile SPI are • Higher commitment than in traditional SPI • Iterations give quick feedback – easier to adopt changes • Increased awareness of the value of process improvement • Which processes are needed and why • What is expected from the process

22. Problems for agile SPI The main problems: • Focus on this project – short sighted SPI • No mechanism for transferring the improvement to the rest of the company • No defined method for checking that the changes have • the intended effects • no unintended side-effects

23. Cost and risk – 1 Cost and risk are related and they spell out the ambition of the SPI activity. If we are • going for scientific and general results => large amounts of data and a statistical analysis. • satisfied with a quick answer which might apply to only the problem at hand=> just a few data and a simple analysis method. We can reduce the risk of a wrong decision by • collecting more data. • take smaller steps On the other hand, postponing the decision of implementing a new and presumably better method and using time for data collections also have their costs.

24. Use of expert knowledge Risk Use of collected data Expert knowledge vs. data

25. Cost and risk – 2 We can reduce the risks by doing small improvement steps. In this way, we can move in small steps with a low risk but also with small effects. The small effects will add up and, in the end, give large improvement. This is for instance the main idea of RPI.

26. Time frame and stability Environmentstability Risk management Low CMM Mean and lean Medium We are moving this way Measurementbased SPI High Company time frame Long Medium Short

27. Clear trends – 1 • More and more companies will adapt all or a considerable part of Scrum. • Availability of free-ware tools will increase the speed of adaptation of new tools. It is easy to try new ways to solve old problems. The investment is small – learn the tool and try it out – and the benefits can be large. • For many application areas, adaptability and flexibility are more important than reliability. It is easy to roll back to a previous version if we make mistakes. • For several application areas, software development is the bottleneck when deploying a new product. This is for instance the case for cars. There will be increased pressure on the software industry to deliver faster.

28. Clear trends – 2 One trend might overshadow the others – outsourcing. • Many Norwegian software development companies are under considerable price pressure. This has lead to outsourcing – first to India and then to Eastern Europe, e.g., Ukraine and the Baltic states. • This lead to a situation where the Norwegian company is responsible for customer contact and high-level design • Development and testing are done through outsourcing to a separate company, a foreign branch or to a foreign partner. The question of SPI will then be concerned with how to select the best outsourcing company or partner.