1 / 52

Process and Open Source Software

This presentation provides an overview of software processes, process modeling, and challenges in software process. It also covers open source software processes and best practices. Includes a case study on a major telecommunications company. Presented by Walt Scacchi from the Institute for Software Research at UC Irvine.

sballenger
Download Presentation

Process and Open Source Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Walt Scacchi Institute for Software Research UC Irvine http://www.ics.uci.edu/~wscacchi/Presentations/Process/Informatix-Process-Lecture.ppt Process and Open Source Software

  2. Overview • Core concerns • Software process and process modeling • Backstory/case study • History of software process • Current challenges in software process • Open source software processes • Best practices

  3. Core concerns • Process(es)‏ • What to do and how to do it • Tasks, inputs, outputs, performers, tools • Performers enact tasks using tools to transform inputs into outputs • Tasks are partially ordered • Tasks, inputs, performers, tools may have pre-conditions • Tasks, outputs, performers may have post-conditions • Organize and coordinate who does what, (with what resources), when, where, how, and why • Formal modeling of processes enables systematic querying and reasoning about complex processes

  4. Core concerns • Process modeling • Understanding, analyzing, and (re)designing software development processes • Models may be either • Prescriptive—what should be done and how to do it • Proscriptive—what might be done and how • Descriptive—what was done and how (historic record)‏ • Models may be either • Narrative • Semi-structured (form-based; hypermedia-based) • Graphic (attributed flow graphs)‏ • Formal (language-based process programs or specifications)‏ • Families of models inter-related through meta-model • Meta-model provides model ontology and process epistemology

  5. Backstory/case study • Major TelCo wants to develop broadband multi-media telecommunications system • Anticipates $1B software development, up to 1500 software developers working 2-3 years • Seeks industrial/IT partners to provide supporting infrastructure to reduce risk • IT partner wants to showcase new “process support technology” products as sales lead • IT partner brings in academic research team to analyze and advise TelCo on “process issues”

  6. Backstory • Academic team, IT partner, and major TelCo jointly elicit, capture, codify (model) and inter-relate TO-BE system development process. • Academic team employs IT partner’s products to present results of their “process analysis” • IT partner considers its process-centered product major success to help large sales engagements • Academic team view of their effort -- a major success for publication, but... P.K. Garg, P. Mi, T. Pham, W. Scacchi, and G. Thunquest, The SMART Approach to Software Process Engineering, Proc. 16th. Intern. Conf. Software Engineering, IEEE Computer Society, Sorrento, Italy, 341-350, May 1994.

  7. A complex organizational process (and model): a decomposition-precedence relationship view (19 levels of decomposition, 400+ tasks)‏ W. Scacchi, Experience with Software Process Simulation and Modeling, J. Systems and Software, 46(2/3):183-192,1999.

  8. Backstory • Academic team suggests overall process will not succeed -- too complex, too much delegation, problematic hand-offs (“throwing it over the wall”)‏ • TelCo and IT partner dismiss academic team • Less than one year later, IT vendor abandons process technology product • Two years later, business press reports TelCo experiences major project failure and losses greater than $200M, and no system.

  9. Software process model history • Process flow charts • Software life cycle (“waterfall” diagram) • Incremental development • Spiral model • Process programs (formal specification) • Capability maturity models and industrial/military standards • Not process models--only focus on what to do, but not how to do it.

  10. Software Development Process Model, c. 1955.

  11. Software Development Process Model, c. 1960.

  12. Software Development Process Model, c. 1970.

  13. Recent software process challenges • Distributed, decentralized, and/or global software development • Process improvement • Process design optimization or redesign • Continuous process improvement (learning) • Articulating software evolution processes • Open source software development processes • Process discovery • (semi-)automated extraction of process events, conditions, timestamps, and other meta-data from software development artifacts

  14. J. Noll and W. Scacchi, Supporting Software Development in Virtual Enterprises, Journal of Digital Information 1(4), February 1999.

  15. Free/Open Source Software Development Processes and Practices

  16. What is free/open source software development? • Free (as in “freedom”) vs. open source • Freedom to access, browse/view, study, modify and redistribute the source code • Free is always open, but open is not always free • F/OSSD is not “software engineering” • Different: F/OSSD can be faster, better, and cheaper than SE in some circumstances • F/OSSD involves more software development tools, Web resources, and personal computing resources

  17. OSS Development Models • Free Software (GPL) • Open Source (BSD/MIT, Mozilla, Apache) • Corporate Source (Hewlett-Packard) • Consortium/Alliance (OSDL, SugarCRM) • Corporate-Sponsored (IBM-Eclipse, Sun-Netbeans, Sun-OpenOffice, HP-Gelato) • Community Source (Sakai, Westwood)

  18. OSSD Project Characteristics • OSS Developers are always users of what they build, while OSS users (>1%) are also OSS developers • Requires “critical mass” of contributors and OSS components connected through socio-technical interaction networks • OSSD projects emerge/evolve via bricolage • Unanticipated architectural (de)compositions • Multi-project component integrations • OSSD teams use 10-50 OSSD tools to support their development work

  19. OSSD Project Characteristics • Operational code early and often--actively improved and continuously adapted • Post-facto software system requirements and design • OSSD is not Software Engineering • OSSD has its own “-ilities” which differ from those for SE • Caution: the vast majority of OSSD projects fail to grow or to produce a beta release.

  20. SourceForge.net

  21. Google Summer of Code 2007

  22. OSS Processes for Requirements or Design • OSS Requirements/Designs • not explicit • not formal • OSS Requirements/Designs are embedded within “informalisms” • Example OSS informalisms to follow (as screenshot displays) • OSS Requirements/Design processes are different from their SE counterparts.

  23. SE vs. OSS processes for Requirements • Elicitation • Analysis • Specification and modeling • Validation • Communicating and managing • Post-hoc assertion • Reading, sense-making, accountability • Continually emerging webs of discourse • Condensing and hardening discourse • Global access to discourse

  24. Evolutionary redevelopment, reinvention, and redistribution in OSS • One evolutionary dynamic of OSSD is reinvention • Reinvention enables continuous improvement • OSS evolve through minor mutations • Expressed, recombined, redistributed via incremental releases • OSS systems co-evolve with their development community • Success of one depends on the success of the other • Closed legacy systems may be revitalized via opening and redistribution of their source • When enthusiastic user-developers want their cultural experience with such systems to be maintained.

  25. Configuration management and work coordination in OSSD • Use CM to coordinate and control who gets to update what part of the system/online artifacts. • Many OSSD projects use CVS (single centralized code repository with update locks) and frequent releases (daily releases on active projects) • Linux Kernel: Git (multiple parallel builds and release repositories) • Collab.Net and Tigris.org: Subversion (CVS++) • Apache: Single major release, with frequent “patch” releases (e.g., “A patchy server”) • GNU arch seeks to develop Free CM unification

  26. Project management and career development in OSSD • OSSD projects self-organize as a meritocractic role-hierarchy and virtual project management • Meritocracies embrace incremental innovations over radical innovations • VPM requires people to act in leadership roles based on skill, availability, and belief in project community • OSS developers value freedom of choice and expression. • want to learn about new stuff (tools, techniques, skills, etc.), • have fun building software, • exercise their technical skill (e.g., developing reusable code), • try out new kinds of systems to develop, and/or • interconnect multiple F/OSSD projects

  27. A meritocractic role hierarchy for OSSD (images from A.J. Kim, Community Building on the Web, 2000)‏

  28. --------------------------- ---------------------

  29. Socio-technical and cultural evolution processes in OSSD • New kinds of OSSD processes under study • Joining and contributing to a project in progress • Role-task migration: from project periphery to center • Alliance formation and community development • Independent and autonomous project communities can interlink via social networks that manipulate objects of interaction • Enables possible exponential growth of interacting and interdependent community as socio-technical interaction network

  30. Discovering the what and how of OSSD processes • Practitioner reflections and anecdotes • Surveys • Ethnographic field work (virtual ethnography) • Mining OSSD project repositories • Multi-modal modeling, analysis, and validation of OSSD processes

  31. Rich Picture Funds, support, Promote Java/Open source Sun Microsystems Download and use free software Share knowledge and ensure all community issues are addressed Ensure that the netbeans community is being run in a fair and open manner Configure and maintain CVS Community Manager Start new release phase, propose schedule/plan respond to tech issues, unanswered questions Release Manager make decisions for the community, on high level download new release The Board Users report bugs release proposal, release updates, branch for current release, release post mortem, review release candidates (2) & decide final release grant access CVS Manager Mailing Lists Manage website Website Tools deploy builds download development builds and test, release Q-builds SourceCast CVS IssueZilla decide features for the project and merge patches/bug fixes, create module web page Site Administrator select feature to develop, bug to fix, download netbeans, commit code QA Team Produce Q- builds and ensure quality of the software Maintain a project/ module, manage a group of developers Contribute to community, meet time constraints for the release grant CVS commit privilege to developers Maintainer Developers/ Contributors Link to all Use Cases Link to Tools Links to all Agents

  32. NetBeans

  33. Formal model of an OSSD process coded in PML(excerpt) • ... • sequence Test { • action Execute automatic test scripts { • requires { Test scripts, release binaries } • provides { Test results } • tool { Automated test suite (xtest, others) } • agent { Sun ONE Studio QA team } • script { /* Executed off-site */ } } • action Execute manual test scripts { • requires { Release binaries } • provides { Test results } • tool { NetBeans IDE } • agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers } • script { /* Executed off-site */ } } • iteration Update Issuezilla { • action Report issues to Issuezilla { • requires { Test results } • provides { Issuezilla entry } • tool { Web browser } • agent { users, developers, Sun ONE Studio QA team, Sun ONE Studio developers } • script { • <br><a href="http://www.netbeans.org/issues/">Navigate to Issuezilla </a> • <br><a href="http://www.netbeans.org/issues/query.cgi">Query Issuezilla </a> • <br><a href="http://www.netbeans.org/issues/enter_bug.cgi">Enter issue </a> } } • …

  34. PML validation analysis

  35. Best practices • Processes with explicit process models are easier to manage, analyze, improve, distribute, and reuse • New/ unfamiliar software tools and techniques are best candidates for software process support • Process meta-modeling enables process life cycle engineering and formal reasoning about processes • OSSD is a community building process • not just a technical development process • F/OSS peer review creates a community of peers • OSSD processes often iterate daily versus infrequent singular (milestone) Software Life Cycle Engineering events • OSSD: frequent, rapid cycle time (easier to improve) vs. • SLC: infrequent, slow cycle time (harder to improve)

  36. Best practices • Easiest to improve a process that is formally modeled • Process management and improvement have been one of the most enduring practices in Software Engineering for improving productivity and quality, and to reducing cost and risks.

  37. Research opportunities

  38. Research opportunities • FOSSD is poised to alter the calculus of empirical SE • Software process discovery, modeling, and simulation • Repository mining can support software visualization, refactoring/redesign studies • Comparison of SE versus FOSSD approaches to software inspection and peer review

  39. Research opportunities • Based on results from individual motivation, participation, role migration, and turnover in FOSSD projects, SE world would benefit from empirical studies that examine similar patterns in conventional software development projects • Is FOSSD more fun, interesting, and rewarding than SE?

  40. Research opportunities • Conventional software cost estimation techniques (e.g., “total cost of operation”) slight/ignore social capital and socio-technical resources • Miscalculation of total resources and capabilities that affect predicted/actual costs of software development or FOSSD

  41. Research opportunities • Results from study of cooperation, coordination and control in FOSSD • Virtual project management and role migration processes can provide a lightweight approach to SE project management • Unclear whether proprietary software projects willing to embrace VPM

  42. Research opportunities • Alliance formation and social networking results suggest SE projects operate at a disadvantage compared to FOSSD projects • SE projects tend to produce systems whose growth/evolution is limited • FOSSD projects can produce systems capable of sustained exponential growth/evolution of both software and developer-user community

  43. Research opportunities • How best to encourage the emergence of a social movement that combines best practices of FOSSD and SE • Consider participation or study of open source software engineering (OSSE) projects at Tigris.org • OSSE seeks to combine SE and FOSSD tools, techniques, and concepts

More Related