1 / 40

Risk management and Process Improvement of Off-The-Shelf Based Development

Risk management and Process Improvement of Off-The-Shelf Based Development. Jingyue Li (jingyue @ idi.ntnu.no ) , Reidar Conradi, Odd Petter N. Slyngstad , Norwegian University of Science and Technology Marco Torchiano, Maurizio Morisio, Dip.Automatica e Informatica, Politecnico di Torino

keely-logan
Download Presentation

Risk management and Process Improvement of Off-The-Shelf Based Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Risk management and Process Improvement of Off-The-Shelf Based Development Jingyue Li (jingyue@idi.ntnu.no), Reidar Conradi, Odd Petter N. Slyngstad, Norwegian University of Science and Technology Marco Torchiano, Maurizio Morisio, Dip.Automatica e Informatica, Politecnico di Torino Christian Bunse Fraunhofer IESE CBSE Seminar -4 Feb 2005- OSLO

  2. Agenda • Research design • Background • Research questions • Sample selection • Results • Selected samples • Answers to research questions • Discussions • Conclusions and future work CBSE Seminar -4 Feb 2005- OSLO

  3. Research design – Motivation • Pre-study background • This study followings a pre-study with 16 structured interviews in Norway, from Oct. 2003 to Feb. 2004. • Focused on SPI in COTS-based development • Respondents shared a lot of experiences on risk management in COTS-based development • Limitations of the pre-study • Small sample size • Sample selected on convenience • Motivation of this main study • State-of-the-practice survey • Randomly selected much larger samples to validate conclusions of the pre-study • Also included Open Source Component CBSE Seminar -4 Feb 2005- OSLO

  4. Research design – research questions • RQ1 - How to improve the development process in projects using OTS components. • RQ2 - How to predict possible risks(problems) in projects using OTS components? • RQ3 - What are the effective methods to mitigate risks in projects using OTS components? • RQ4 - What are the similarities and differences between projects using COTS and OSS components? CBSE Seminar -4 Feb 2005- OSLO

  5. Research design – sample selection • Norway • Germany • Italy (Sample selection reported in later presentation) CBSE Seminar -4 Feb 2005- OSLO

  6. Research results – selected samples • Current data • Total 86 projects • Norway • 46 projects from 38 companies • One company filled in 4, one filled in 3, and one filled in 2. • In other companies, we selected only one project each company • Germany • 29 projects from 29 companies • Italy • 11 projects from 11 companies • Data collection is still on-going in Germany and Italy CBSE Seminar -4 Feb 2005- OSLO

  7. Research results – selected companies CBSE Seminar -4 Feb 2005- OSLO

  8. Research results – selected companies (cont’) Small (0-19) Medium (20-99) Large (more than 100) CBSE Seminar -4 Feb 2005- OSLO

  9. Research results – selected projects CBSE Seminar -4 Feb 2005- OSLO

  10. Research results – selected respondents CBSE Seminar -4 Feb 2005- OSLO

  11. Research results – selected respondents (cont’) • 85% respondents have more than 3 years experience on OTS-based development • Most respondents have the Bachelor degree in informatics, 10% have Ph.D degree. CBSE Seminar -4 Feb 2005- OSLO

  12. Research question RQ1 • How to improve the development process in projects using OTS components? • Overall development process • Do I need to change my main development process dramatically in projects using OTS? • What activities and roles should be added? • OTS selection process • Formal decision making process? • Familiar with component process? CBSE Seminar -4 Feb 2005- OSLO

  13. RQ1: Do I need to change my main development process dramatically? • More than 80% projects members decided their main development process (Waterfall, incremental, etc.) before they started to think about using OTS. • It actually worked. CBSE Seminar -4 Feb 2005- OSLO

  14. RQ1: What should be added? • Activities • ”Acquire” vs. ”build” decision • OTS component selection • Learning OTS component • Build glueware and/or addware • A new role (OTS knowledge keeper) • Germany (100%) • Norway (37%) • Italy (9%) CBSE Seminar -4 Feb 2005- OSLO

  15. RQ1: What is the proper OTS selection process? • Formal decision making process (by 15% used) • Selecting evaluation criteria (factors) • Collecting and assigning values to these criteria • Applying formal decision making algorithms such as MAUT or MCDA etc. • Familiar with component process (by 85% used) • Search internet • Limited to 2-3 components • Download demo version and try it, then decide Or • Recommended from internal/external experts CBSE Seminar -4 Feb 2005- OSLO

  16. Research question RQ2 • How to predict possible risks in projects using OTS components? • What were the most frequent risks (problems) in practice? • Was there any relationship between those risks (problems) and the project profile? CBSE Seminar -4 Feb 2005- OSLO

  17. RQ2: Typical risks CBSE Seminar -4 Feb 2005- OSLO

  18. RQ2: Typical risks (cont’) CBSE Seminar -4 Feb 2005- OSLO

  19. RQ2: Frequency of typical risks (problems) in OTS based development CBSE Seminar -4 Feb 2005- OSLO

  20. RQ2: Frequency of typical risks in OTS based development (cont’) • Most frequent risks • Effort to integrate OTS components was not satisfactorily estimated • Keep up with requirements evolution • Identify defects inside or outside OTS component • Least frequent risks • Negative reliability effect • Negative security effect • Negative performance effect • Lack provider information CBSE Seminar -4 Feb 2005- OSLO

  21. RQ2: Relationship between typical risks (problems) and project context • The more different OTS-components used in the project, the more frequent the following risks: • Identify whether defects were inside or outside the OTS components • It was difficult to update the system with the last version OTS components • Provider did not provide enough technical support/training CBSE Seminar -4 Feb 2005- OSLO

  22. RQ2: Relationship between typical risks (problems) and project context (cont’) • The higher the general experience on OTS-based development in projects, the less frequent the following risks: • Effort to integrate OTS c components was not satisfactorily estimated • It was difficult to identify whether defects were inside or outside the OTS components CBSE Seminar -4 Feb 2005- OSLO

  23. RQ2: Relationship between typical risks (problems) and project context (cont’) • The project with an OTS knowledge keeper had less frequency on the following risks than project without OTS knowledge keeper: • Difficult ot identify risks inside or outside OTS components • Lack the information of the vendors’ reputation and support ability CBSE Seminar -4 Feb 2005- OSLO

  24. Research question RQ3 • What are the effective methods to mitigate risks in projects using OTS components? • Which strategies had been frequently used in practice? • What were the effective strategies? CBSE Seminar -4 Feb 2005- OSLO

  25. RQ3: Proposed risk management strategies • Customer had been actively involved in “acquire” vs. “build” decision • Customer had been actively involved in OTS component selection • OTS components were selected mainly based on architecture and standards compliance, instead of expected functionality • OTS components qualities (reliability, security etc.) were seriously considered during selection • Effort in learning OTS component was seriously considered in effort estimation CBSE Seminar -4 Feb 2005- OSLO

  26. RQ3: Proposed risk management strategies (cont’) • Effort in black-box testing of OTS components was seriously considered in effort estimation • Unfamiliar OTS components were integrated first • Did integration testing incrementally (after each OTS component was integrated) • Local OTS-experts actively followed updates of OTS components and possible consequences • Maintained a continual watch on the market and looked for possible substitute components • Maintained a continual watch on provider support ability and reputation CBSE Seminar -4 Feb 2005- OSLO

  27. RQ3: Frequency of using proposed risk management strategies in practice CBSE Seminar -4 Feb 2005- OSLO

  28. RQ3: Frequency of using proposed risk management strategies in practice (cont’) • The most frequently used risk management strategies: • OTS components qualities were seriously considered in the selection process • Unfamiliar OTS components were integrated first • Did integration testing incrementally • Local OTS-experts actively followed updates of OTS components and possible consequences • The least frequently used risk management strategies: • Involve customers in the “acquire” vs. “build” decision • Invove customers in OTS selection CBSE Seminar -4 Feb 2005- OSLO

  29. RQ3: What were effective risk management strategies ? CBSE Seminar -4 Feb 2005- OSLO

  30. RQ3: Risk management recommendations in OTS-based projects • Avoid risk • Do not use too many different OTS components in one project • Manage risk • Manage the knowledge of OTS properly (Have a OTS expert and share OTS experience regularly) • Spend enough time on OTS quality evaluation. Hand-on trial is necessary • Do not marry specific OTS. Be ready for possible replacement • Maintain a continual watch on provider support ability and reputation CBSE Seminar -4 Feb 2005- OSLO

  31. Research question RQ4 • What are the similarities and differences between projects using COTS and OSS components? • Are there any similarities and differences in: • Company, project, system profile ? • Motivation of using them ? • Frequency of risks (problems) ? CBSE Seminar -4 Feb 2005- OSLO

  32. RQ4: Selected samples – COTS projects vs. OSS projects • 56 projects used only COTS • 25 projects used only OSS • 5 projects used both COTS and OSS (not considered in data analysis) CBSE Seminar -4 Feb 2005- OSLO

  33. RQ4: Are there any similarities and differences in company profile ? CBSE Seminar -4 Feb 2005- OSLO

  34. RQ4: Are there any similarities and differences in company profile ? (cont’) CBSE Seminar -4 Feb 2005- OSLO

  35. RQ4: Are there any similarities and differences in project profile ? CBSE Seminar -4 Feb 2005- OSLO

  36. RQ4: Are there any similarities and differences in system profile ? CBSE Seminar -4 Feb 2005- OSLO

  37. RQ4: Are there any similarities and differences in company, project, and system profile ? • Our conclusion • There is no difference in company, project and system profile between projects using COTS and OSS. CBSE Seminar -4 Feb 2005- OSLO

  38. RQ4: Are there any similarities and differences inmotivation of using COTS vs. OSS ? • Commonalities • Shorter time-to-market • Less development and maintenance effort • Higher reliability • Differences • COTS • Follow the market trend • Paid software will give good reliability • Good support • OSS • New technology • Free source code • Avoid the risk in OSS evolution CBSE Seminar -4 Feb 2005- OSLO

  39. RQ4: Are there any similarities and differences in frequency of risks (problems) ? • Commonalities • Requirement changed a lot and it was difficult to keep up with these changes • Differences • COTS: higher risk on following evolution of both requirements and COTS component • OSS: higher risk on getting good support CBSE Seminar -4 Feb 2005- OSLO

  40. Questions ? CBSE Seminar -4 Feb 2005- OSLO

More Related