Classical Open Source Software Process Model

1. Classical Open Source Software Process Model Alan Kelon Oliveira de Moraes alan@kelon.org Recife, November 27, 2006

3. Open-source software is not a new idea but only recently have technical and market forces converged to draw it out of a niche role. Eric S. Raymond The Cathedral & the Bazaar, 1999

4. open source companies had raised ~$900 million from venture capitalists since 1999 [Rivilin 2005]

5. Summary • What is open source and why should you care about it? • Classical open source process model • Remarks • References

6. Definitions and Motivations What and why?

7. What is Open Source? • Open Source software is the process of systematically harnessing open developmentand decentralized peer review to lower costs and improve software quality [Raymond 1999]

8. Basic idea behind open source • When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. • People improve it, people adapt it, people fix bugs. http://www.opensource.org [Open Source Initiative]

9. Open source license definition • Open source means more than source code availability: • The source must be available for redistribution without restriction and without charge • The license must permit the creation of modifications and derivative works • Must allow those derivatives to be redistributed under the same terms as the original work. [OSI] [Perens 1999] [O’Reilly 1999]

10. How is open source different? • The term open source generally refers to software that differs from its commercially sponsored counterpart in three ways • source code’s ready availability to prepare derivative works • collaborative production approach • community governance of production [West 2005]

11. It is still evolving... • The term “open source project” does not have a clearly defined stable and shared meaning comparable to “open source license” • The processes evaluated refer to “classical” open source projects [O’Mahony & West 2005]

12. Why should you care? • It’s important not to reduce the open-source debate to a question of NT versus Linux, or Microsoft versus the rest of the world • Understand how the lessons of open source can be applied to software development across the board. • The open-source process reflects a powerful global trend toward networked collaboration. • Printing press: spread of knowledge in Renaissance • Open Source: large scale cooperative development efforts today. [O'Reilly 1999]

13. Open wallets for open source software • In 1999 and 2000, according to VentureOne, venture capitalists invested $714 million in 71 open-source companies • Turbolinux raised $95 million • Linuxcare raised at least $80 million • Most of those projects collapsed… [Rivilin 2005]

14. Open wallets for open source software • Venture capitalists are embracing open-source technology companies again • $149 millions to 20 business in 2004 • At least three open-source start-ups raised $20 million in April 2005 • The big difference is the increased adoption of open-source software by corporate users today • Red Hat had $125 million in revenue in 2004 and now has a market capitalization around $2 billion. [Rivilin 2005]

15. New business models • JBoss's business model, built on selling support services, made sense [Rivilin 2005] • Open Source has effect on commoditization, system architecture and network effect, and the development practices associated with software as a service[O'Reilly 2005]

16. Classical Open Source Process Model A brief survey

17. Hey, wait! What is a software process? • A particular method of doing something, generally involving a number of steps or operations [Webster dictionary] • The software process consists of the activities and associated information that are required to develop a software system [Sommerville 1996] • A process defines specifically who does what, when, and how [Fayad 1997] • Software process is a set of activities, methods, practices, and transformations which people use to develop and maintain software and the associated products [Paulk et al. 1993]

18. And what is a software process model? • While a process is a vehicle for doing a job, a process description is a specification of how the job is to be done [Osterweil 1987]

19. Abstract generic process models • Specification-based models • Waterfall • Incremental • Evolutionary development models • Prototyping • Spiral [Sommerville 1996]

20. The Cathedral and the Bazaar • Development style of Linux • Release early and often, and listen to your customers • Delegate everything you can • Be open to the point of promiscuity • Present a plausible promise [Raymond 1999]

21. Brooks vs. Linus • Brooks’ Law: ‘‘Adding more programmers to a late project makes it later.’’ • Linus’ Law: “Given enough eyeballs, all bugs are shallow” [Raymond 1999]

22. A case study of open source software development: the Apache server • Apache Server Process: decision making-oriented process • Emphasized decentralized workspaces and asynchronous communication • Email lists exclusively to communicate with each other, and a minimal quorum voting system for resolving conflicts [Mockus et al. 2000]

23. A case study of open source software development: the Apache server • There is no explicit system-level design, or even detailed design • There is no project plan, schedule or list of deliverables • In order to keep track of the project status, an agenda file is stored in each product's repository, containing a list of high priority problems, open issues, and release plans. [Mockus et al. 2000]

24. A case study of open source software development: the Apache server • There is no single development process, but these are developer's common actions: • Discovering that a problem exists • Determining whether a volunteer will work on it • Identifying a solution • Developing and testing the code within their local copy of the source • Presenting the code changes to the group for review • Committing the code and documentation to the repository. [Mockus et al. 2000]

25. A case study of open source software development: the Apache server • Quantitative analysis based on Developer mailing lists, CVS commit logs and Problem reporting database • From may February 1995 to May 1999 [Mockus et al. 2000]

26. A case study of open source software development: the Apache server • Almost 400 individuals contributing code • The top 15 developers contributed more than 83% of the MRs and deltas, 88% of added lines and 91% of deleted lines. [Mockus et al. 2000]

27. A case study of open source software development: the Apache server • Around 3060 different people submitted 3975 problem reports. • The top 15 problem reporters (only 3 also in the core) submitted only 213 or 5% of Prs • Almost 2600 developers submitted one report, 306 submitted two, 85 submitted three • The maximum number submitted by one person was 32. [Mockus et al. 2000]

28. A case study of open source software development: the Apache server • Time to resolve problem reports • 50% of PRs are resolved within a day • 75% within 42 days • 90% within 140 days [Mockus et al. 2000]

29. Descriptive Process Model for Open Source Software Development • Three basic views for software process models [Humphrey 1989] • The state view • The organizational view • The control view [Johnson 2001]

30. Descriptive Process Model for Open Source Software Development • State view • Closed prototyping • Iterative and incremental enhancement • Concurrent development • Large-scale peer review • User-driven requirements [Johnson 2001]

31. Descriptive Process Model forOpen Source Software Development • Organizational view • Decentralized collaboration • Trusted leadership • Internal motivation • Asynchronous communication [Johnson 2001]

32. Descriptive Process Model for Open Source Software Development • Control view • Informal planning • Tiered participation • Modular design • Ubiquitous tool support • Shared information space [Johnson 2001]

33. An overview of the Software Engineering Process in the Mozilla project • The importance of tools • Bugzilla • Tinderbox • Bonsai • LXR [Reis and Fortes 2002]

34. An overview of the Software Engineering Process in the Mozilla project • Great deal to develop and to document the Mozilla software process • Rely heavily on communication tools such as mailing lists, irc, news, bugzilla • “Bug-driven” development {bugzilla} • Requirements • A message thread is started on a public newsgroup regarding a change in functionality • When a person has a specific idea for a change a bug will often be directly filed with no newsgroup discussion • “You can get a feature in if you’re willing to write code for it” -- Boris Zbarsky, Mozilla developer • Code review and inspection {bonsai} • Nightly build {tinderbox} [Reis and Fortes 2002]

35. An overview of the Software Engineering Process in the Mozilla project • Effective version control • A well-defined protocol for integrating source code changes • A high degree of accountability for people who integrate code • High modularity • Good communication channels • Documentation not always up to date [Reis and Fortes 2002]

36. Traditional SE: Elicitation Analysis Specification and modeling Validation Communicating and managing Open Source Post-hoc assertion Reading, sense-making, accountability Continually emerging webs of discourse Condensing and hardening discourse Global access to discourse Understanding the Requirements for Developing Open Source Software Systems [Scacchi 2002]

37. Free Software development lifecycle [Senyard and Michlmayr 2004]

38. Bazaar style development lifecycle [Senyard and Michlmayr 2004]

39. Quality assurance under the open source development model • User participation in open source projects is high • up to 20% of the changes for almost 50% of the projects • discovering 20–40% of the faults in 20% of the projects. • Change management processes and tools are at the cutting edge of large-scale collaborative software development • Documentation is not a high priority for most projects: ‘‘TODO’’ and install guide • Little investment in utilizing testing techniques and tools, relying mainly in community testing [Zhao and Elbaum 2003]

40. Open source software maintenance process framework [Koponen and Hotti 2005]

41. Open source software maintenance process framework [Koponen and Hotti 2005]

42. Concluding remarks Past, present and future

43. Past • Hacker community activity • Self-contained moviment

44. Present • Open source gets momentum • Developing F/OSS is different than software engineering [Scacchi] • not better, not worse, but different and new • more social, more accessible • F/OSS systems don’t need and probably won’t benefit from classic software engineering [Scacchi]

45. Abstract generic process models • Specification-based models • Waterfall • Incremental • Evolutionary development models • Prototyping • Spiral • Open Source* [Sommerville 1996]

46. So an open source process model is… • Evolutionary/maintenance-oriented development model • Not just a technical development process • Collaborative effort • Based on community government • User-driven requirements • Modular architecture/design without UML stuff • Heavily dependent on communication tools because it is distributed, asynchronous and discontinuous • Indeed it is not a traditional software engineering process

47. Future • Open source is still in its infancy today • OSS projects need to be more professional • Open Source Software Process • Do not to move into Process Paralysis [Yourdon 1987]

48. http://www.fabricadesol.com/in953

49. [Neus et al. 2005]

50. [Neus et al. 2005]