Simulating for Software Engineering using Systems Dynamics

1. Simulating for Software Engineering using Systems Dynamics Jenny Longster Martin Shepperd Brunel University, BT

2. Contents • Introduction to Simulation • Simulation Approach • Introduction to System Dynamics • Basic Modelling Building Blocks • Simulating Software Projects • Putnam Model of Software Lifecycle • The BT Research Project • Introduction to COTS • The COTS Requirements Model • Results • Discussion and Future Work

3. Introduction to Simulation • Definitions • System • a system exists and operates in time and space • Model • simplified representation of a system at some point in time or space • intended to promote understanding of the real system • Simulation • manipulation of a model so that time is compressed • behaviour of the system under specific conditions may be studied • enables interactions to be seen in a matter of minutes • Dynamic complexity • systems have cause and effect • today’s systems more and more complex

4. Simulation Approach • Two main approaches • Discrete modelling • Continuous modelling • Discrete event modelling • time advances when a discrete event occurs to the next event time • e.g. traffic – car arrives • an associated action takes place • implies placing a new event in an event queue • difficult to monitor continually changing variables - clunky • Continuous Simulation based on System Dynamics • time is controlled by continuous variables expressed as differential equations • see the interactions between key process factors • considered to be better at showing qualitative relationships • method chosen

5. Introduction to System Dynamics • Used to solve real world problems • pressing business/process issues • Systems Thinking • the ability to see the world as a complex system • understand that “you can’t do just one thing” and that “everything is connected to everything else” • System Dynamics • exponential growth of technology and complexity • helps understand complexity • design better operating policies • guide changes in systems

6. Introduction to System Dynamics • Seeking to solve a problem often makes it worse • Policies create unanticipated side effects • Attempts to stabilize systems may destabilize them further • Decisions may provoke reactions by others seeking to restore the balance upset • Examples of adverse reactions to policies • Low tar cigarettes • Pesticides and herbicides • IT has not enabled the “paperless office” • Antilock brakes cause people to drive more aggressively

7. Basic Modelling Building Blocks:iThink Stocks Purpose: Stocks are accumulations. They collect whatever flows into and out of them Flows Purpose: to fill and drain stocks. The arrow indicates the direction of the flow Converters Purpose: converts inputs to outputs - can hold values for constants, define external inputs to the model, perform algebraic calculations Connectors Purpose: connects model elements

8. Example Model: Publication Output • Consider a university with 120 people active in research • 120 people composed of 60 PhD researchers and 60 professors • University wishes to maintain the total number of people active in research at 120 • Professors quit at a rate of 10 per year • One new PhD student hired for each professor who quits • PhDs don't quit! • 6 years to develop a professor from a PhD • PhDs produce 3 papers per year • Professors produce 10 papers per year • All 120 active researchers are fully active (!)

9. iThink Model Initial Professors = 60 Initial Researchers = 10, 10, 10, 10, 10, 10 Quits = 10 Hires = quits xfer = 6 years Publication rate = 3 * researchers + 10 * professors

10. iThink Model Results stable publication output, stable number of profs, PhDs

11. Management Policy change • suppose the university introduces a new policy • Professors must clean toilets as part of their job description • Annoys professors • Quit rate moves from 10 to 15 a year

12. Results of Policy Change • in the 10th year university management notices its publications have dropped from 780 per year to 570 per year • it wonders what has happened • where does it look for the problem? • around the 10th month • the university finds that it still has 120 active researchers, yet there are now 30 professors and 90 PhDs • A most puzzling situation! • The university, with it's built in hiring rule, essentially an auto pilot no thought action, hired one PhD for each professor that quit • this one time transition in quit rate set off a 6 year transition within the university • leading to a new equilibrium state with 30 professors and 90 PhDs

13. Results of Policy Change Publication rate dropped from 780 to 570 per year

14. Simulating Software Projects • Two models • General Cost Estimation Model • Model specifically for BT project • General Cost Estimation Model • Micro model • bottom up • capture data on small software parts e.g. technology, user changes etc. • COCOMO • Used to estimate desired values - effort, time • Conjecture – applicable outside the organisation? • Macro model • Top down • only use most important large variables– project cost, time, size • Norden/Putnam model • Classic non-linear model • See relationship between schedule, manpower and size – not direct trade off • Conjecture

15. Norden Manpower Distribution Model • Theory of project modelling • Project seen as set of problems to be solved • Development ends when all problems are transformed into solutions • Requires manpower, time and creativity • Manpower effort , K (man.years) is an indicator of number of problems to be solved • Manpower available=manpower required • development optimised in terms of time and cost Manpower Distribution

16. Putman Model of the Software Lifecycle • Norden model extended by Putnam to model software lifecycle • Formal model for software development estimating • Assumed software development follows Norden manpower distribution • With noise – changing requirements etc • Data collection • 1970s, 200 projects • Verified manpower distribution • Peak manning near delivery time • Example usage • Manpower Effort K=95 man.years, delivery time 1.75 years • Can calculate • number of people at peak (33 persons) • Average rate of team build-up (mo/td) • 1.5 person/month - reasonable

17. Difficulty • Slope of manpower distribution at start time t=0 • Called Difficulty D • Larger value shows project is more difficult to manage • Manpower demand is high • Schedule is short • Higher number of people to be integrated and done quickly • Putnam observed Difficulty with 200 projects • Developed a derivative called manpower build-up related to the nature of the development • =8 Entirely new software, complex • =15 New stand alone system • =27 Rebuilt from existing software • Larger value = steeper curve

18. iThink Putnam Model Interface Project manger inputs Effort remaining to be employed (man.years) Delivery time Nature of Development

19. iThink Putnam Model

20. Simulating Software Projects: Putnam Model Results

21. Simulating Software Projects: Cutting Development Time

22. Simulating Software projects: Changing the Nature of the Development

23. Second Model: The BT Research Project • Software industry has undergone significant change • from writing all code in house • to system building with existing component products • Integration of Commercial-Off-The-Shelf (COTS) products • COTS projects have specific features that add new factors • need to be addressed for successful software development • Conducted a systematic review • real-world COTS projects • evidence-based • extracted success and failure factors • Package up Results • simulation of effects of success / failure factors • model the requirements stage of a COTS-based project

24. iThink COTS Requirements Model

25. COTS Requirements Model Interface

26. COTS Requirements Model Interface: Project Properties

27. COTS Model: Acquisition Factors

28. COTS Model: Requirements Success Factors

29. COTS Requirements Model: Results

30. Lower Success Factors and Decreasing Schedule

31. Conclusions • Learning curve about COTS projects • Powerful, flexible tool for analysis • Business visualisation tool – BT • Packaging up mathematical ideas • Putnam model • Packaging up Systematic review results

32. Discussion and Further Work • Extend COTS requirements model to whole lifecycle • Calibrate models • Data? • Ask people to rate importance of success factors? • Is the Putnam model applicable to COTS projects? • Can we make it so? • Monte Carlo Simulation • Single point solution (e.g. calendar months) represents average (50% probability) based on certainty of assumptions • Equally likely to under-run as over-run • For any other probability assign random values and simulate • Enables a ‘best-bid’ solution