Unsustainable Software Development and its Causes

1. Unsustainable Software Development and its Causes Lecture 1CIS 6101 – Software Processes and Metrics

2. Outline • 1. Background of Unsustainable Development • 2. Causes of Unsustainable Development • slides 10-17 • 3. Project Controls - Slide 18 • Plan Driven Development – slide 21 • Project Controls – slide 26 • 4. Summary • slide 33

3. 1. Background of Unsustainable Development

4. Definition • Unsustainable development is characterized by constantly increasing the cost of change to the software. • Evidence of a high cost of change is a constantlyincreasing number of defects. • How does this happen? <Discuss> • Each change • adds complexity and • uncovers or causes of defects that require more changes. • This complexity leads to a declining ability to respond to customer requests. • Software becomes harder and more fragile to maintain.

5. Fixes versus New Features • Unsustainable development is all too-common today in software industry. • We place too much emphasis on • short term feature development (new) and • fixing defects (existing) and • NOT paying attention to health of the underlying software. • Result: Cost of change rising with new changes coupled with the risk where each change can destabilize the product. • <Discuss: What does it mean to ‘destabilize’ the product?>

6. Fixes and Stability of Software • Here, teams spend more and more time fixing defects and in stabilizing software. • Features • Do get developed, but • Less time available due to time for stabilizing the software. • Development teams want to freezerequirements • But this reduces ability to respond to client requests. • Newer methodologies embrace customer involvement!!

7. When is the Threshold Reached – • When team spends more time on defect backlog then developing new features. • When: occur within a few weeks, months, years. •  Finger pointing and blame game. • Consequences: • Morale down; • job layoffs; • outsourcing; • poor customer satisfaction. • Most teams, however, do not live with unsustainable software but get uncomfortably close. • <Discuss: CMDS; the methodology (process); reactions!>

8. TechnicalDebt • Technical debt is the underlying cause of inability to develop new features due to defect burden. • Cause of Technical Debt: • caused by build up of poor decisions over time • decisions that seemedright but often made for short-term fixes. • <Discuss: How does this happen?> • Rarely a single decision is the cause.

9. The Perils of Jumping in Place • Evidence of technical debt is clear. • When products must be rebuilt to keep up with major competition, or • When products need to be rebuilt to respond to major technology changes. • Developed systems last year for COJ only to discover that they used an older version of .NET and an upgrade was not in sight. We went to an older version. They have still not loaded the software! • Rewrites require • massive investments in time and effort and • often results in loss of market share while the replacement product is being built. • Overseas example: OPREP5 (Operational Report 5) • Often impossible to recover market share where competitors get stronger and new competition enters marketplace.

10. 2. The Causes of Unsustainable Development

11. 2. The Causes of Unsustainable Development (10-17) • Why is unsustainable development so common? • A number of reasons called Product Stresses • There are things we cannot control (project stresses) and, • There are a number of things we can control (project controls). • Different for every project; stresses change • <Discuss Stresses without looking ahead…> • Let’s look:

12. Causes of Unsustainable Development – External Dependencies • Dependencies on versions of the OS, compiler, IDEs, web browser components, etc. • Impossible to know when some of these things will occur. • Must often develop to support both old and new interfaces, such as two different versions of IE, netBeans, etc… • Sometimes different clients require different versions of the same software – paying clients.

13. Causes of Unsustainable Development - Competition • Very easy to set up a software development company nowadays. • Need few bright people, a few computers, and Internet • Many do not have • a workable business plan and • a clear idea of their true market worth due to overestimating • do not know their marketsize. • Almost always suffer from inability to adequately sellor market their work. • Usually get initialfunding and make some initial sales, • but struggle to stay in business.

14. Causes of Unsustainable Development - Disruptive Technologies • Constant advances in state of the technology can be disruptive. • Products need to meet users’ demands over the long term. • Cannot restrict thinking for ONLY what the customer wants, because customers may initially be interested in immediate needs. • But customers may be offered morefeatures on newertechnology for lessprice • Example: The Internet is a disruptive technology • more and more support can easily be offered over the Internet. • Companies must learn how to deal with the Internet and its threatstoo (illicit software, hackers, a forum for malicious people to exploit security flaws, and more.)

15. Causes of Unsustainable Development Disruptive Business Models (1 of 2) • ConsiderDell: They have a disruptive business model. • Their model allows them to produce computers on demand • By clustering their suppliers together and • By taking all orders over the Internet. • Other companies (HP, Compaq, IBM) have not been able to keep pace or duplicate Dell’s business. • Open Source software. A disruptive role! • Open source is ‘free,’ and usually usable by highly technical people. • No question that open source will play an increasingly disruptive role in the software industry

16. Causes of Unsustainable Development - Disruptive Business Models (2 of 2) • Microsoft, while having a monopoly on desktop market, cannot compete against free software. • Linux: can its business model be sustained by keeping itself going almost exclusively through support and services as claimed?

17. Causes of Unsustainable Development - Cost Management (last) • Software Projects are exceedingly expensive! • Best way to control costs: employ less expensive people. • This explains the increasing use of outsourcing. • But efficiency is at the heart of sustainable development.

18. 3. Project Controls (18-20) There are things we can do and we can control.

19. Project Controls – Collaboration • Must work with other people w/ithe organization • May involve others • with similar development projects or • With a history of similar projects • Must work with your customers!! • May involve end-user representatives on staff

20. Project Controls – Methodology • Development methodology – the principles (mindset) and practices of planning, design, testing, coding, etc.) has a dramaticimpact on the success of a project. • Methodology directly influences • efficiency, • change tolerance, • responsiveness, and • software quality. • <Discuss these bullets> • Let’s look more closely at Plan Driven Development

21. Methodology – Plan Driven Development (1 of 5) • Plan-driven methodology (which works well sometimes) because this is what is taught in school especially to engineering. • Waterfall model primarily; documentation intensive; rigid time lines, requirements known up front; little flexibility for change; minimal customer contact, late breakage, etc. • Code then Fix: If this doesn’t work, they may use a variant such as code-then-fix approach. • Here, teams spend lots of time writing software and very little time on planning and analysis. (Extreme Programming?) • Software is brittle and will resist change. • Lack of sound practices leads to a great deal of dead code and thrashing.

22. Methodology – Plan Driven Development – 2 of 5 • Plan-driven development, derived from engineering, is best typified by the ISO 9000 set of standards. <Good research topic> • Downside of plan-driven approach (so many…) • some software developers feel all the complexities of the software can be understood before any code is written. • This is rarely true and definitely false over the long term. • <Discuss: When might this be true?>

23. Plan Driven Development – 3 of 5 • Working Thru the Process: While documentation of requirements, design, test plans, etc. hasvalue as a means of communication, their realvalue comes not from the documentation but rather from the process one must go through to producethedocumentation. • Working though a design with your peers is a • valuable exercise; • fosters collaboration, understanding, and creativity; • But: The document is of less value and must be maintained. • <Discuss: is the document used?> • <Discuss: “must be maintained?”>

24. Plan Driven Development – 4 of 5Negatives on Documentation • Work gets done; but results in hugetime spent producing and maintaining the documents. • Thisdetracts from amount of time spent on analysis. • On the other hand, if ‘maintained’ these represent a living, real set of specs, design, maintenance manuals… • Plan-driven (total, fine-grained) software does not work for software development because it is impossible to put together a plan when the project starts to take into account all the changes in requirements and in the environment that will occur over the course of the project. <Discuss: Is this really true??>

25. Plan Driven Development – 5 of 5 • Plan-driven approach stiflesagility, because • They emphasizeceremony and • Assert documentation over having a workingproduct in the erroneous belief that somehow having a pile of documentation describing requirements and design is proof that: • That the project cannot fail, • that a discipline is in use, and by extension, • That a project without the depth of documentation does not have discipline. • What is really needed is/are working projects in user’shands such that feedbackisproduced, since this is the only real way to know if the desired system meets user needs.

26. Project Controls – Expertise (1/7) • This will be a strongindicator of success. • Will be able to understand what is really important and what is not. • What are the indicators of success? • How do we communicate this? • Through whom?

27. Project Controls – Decision Making (2/7) • The ability to make decisions in • a complex and in constantly changing environment • often with incomplete information is crucial to success. • An important component of this is prioritization, where you chose what must be worked on and when. • Who determines this prioritization??? Discuss

28. Project Controls – Leadership (3/7) • Helps to set up and communicate a vision, strategy, and tactics. • Keep projects on track. • Strong leadership is essential!! • What makes a strong leader? Discuss

29. Project Controls – Culture (4/7) • Organizational culture shapes the attitude and practices of the people who work on a project. • What does this mean? • Culture defines the requirements • for leaders, and • for how decisions will be made; by whom, and methodology.

30. Project Controls – Simplicity and Reliability – 1 of 3 (5/7) • Most software today is • overly complex, • hard to use and learn, and • has too many features for most of its users. • Think about word processors: – have features that most users will never use. • Interesting: Word2007 had 265 functions; • only 12 were used by more than 75% of its users; • 42 were not used at all! • Word 2013?? • Ill-effects of unnecessary complexity on users? Hard to say. •  Complexity is behind at least part of the outsourcing trend because one way to manage complexity is to get someone else to do it for you.

31. Project Controls – Simplicity and Reliability – 2 of 3 (6/7) • Huge disparity between what consumers want and what companies would liketo sell them. • Too many companies still compete on features and price alone. • Often encouraged • by the most technical users who are the early adopters of any technology and • by the media, who still publish reviews that contain feature comparison charts.

32. Project Controls – Simplicity and Reliability – 3 (7/7) • Related to simplicity is the need for reliability. • Too many software organizations emphasize features over reliability, as many users refuse to buy the first version of any software, and • Users are frustrated by versionupgrades that breakorremovefeaturesrelied upon in previous version. • Consider Windows 7 and Windows 8. Discuss. • There’s not enough emphasis placed on simplicity and reliability. • Consider the iPod or the Palm PDA. Successful because they did few things very wellseveral years ago! • There was a great deal of restraint in their development. • Easy to add features, but difficult to show restraint to keep them out.

33. 4. Summary – 1 of 2 • Unsustainable development is a development pace that is typified by • stress, • frustration, and • a sense of not being in control. • Evidenced by continually • increasing the cost of change and defect rate and • corresponding decreasingability to respond to changing conditions. • Unsustainability may not be apparent to the people on the project due to the varying pace of decline. But it is there! <Discuss: It is really there. No time to see it> • Some comes from continual change and stresses that are out of control of the organization.

34. Summary – 2 of 2 • Let’s look at the principles and practices and outline some key principles that leads to sustainability. • We have the principles,and these are used to frame the practices that reinforce them.