40 likes | 193 Views
Systems Engineering Effectiveness Committee SE Effectiveness Study Redux. NDIA SE Division Meeting 18 August 2010. History. For the projects that did the least SE, only 15% delivered the best project performance. SE Effectiveness Study conducted by NDIA in 2006-2007
E N D
Systems Engineering Effectiveness CommitteeSE Effectiveness Study Redux NDIA SE Division Meeting 18 August 2010
History For the projects that did the least SE, only 15% delivered the best project performance. • SE Effectiveness Study conducted by NDIA in 2006-2007 • Surveyed 46 programs to identify relationships between SE activities and project performance as measured by cost, schedule, and requirements conformance. • Conducted anonymously to encourage participation and maintain confidentiality • Found strong relationships between Project Performance and IPT, Architecture, Trade Studies, and Requirements activities. PROJECT PERFORMANCE vs. TOTAL SE CAPABILITY 1.00 Best Performance ( x > 3.0 ) Moderate Performance ( 2.5 x 3.0 ) Lower Performance ( x < 2.5 ) 12% 56% 15% 59% 46% 0.75 0.50 13% 39% 0.25 31% 29% For the projects that did the most SE, 56% delivered the best project performance 0.00 Lower Capability ( x 2.5 ) N = 13 Moderate Capability ( 2.5 < x < 3.0 ) N = 17 Higher Capability (x 3.0 ) N = 16 Gamma = 0.32 p = 0.04 Report available at http://www.sei.cmu.edu/library/abstracts/reports/08sr034.cfm
Recent Events • Briefed Mr. Welby on 06-May on the results of the study • Recognized the value of the study • Wanted a wider sample to confirm results • Formed consortium with IEEE Aerospace and Electronics Society to expand study to a wider audience • Committee chaired by Bill Lyons (Boeing) • Support from OSD (Mike McLendon) • Support from USAF (AFMC/EN) • Technical support from SEI
Current Efforts • Developing a research proposal for future work 1: Identify SE activities linked to improved program performance via survey of defense suppliers 2: Assist OSD in updating policy, guidance, training, and tools to deploy proven practices in acquisition programs 3: Assist OSD in establishing a means of routinely collecting data from future programs to facilitate continuous improvement • Preparing an update to the prior survey • Improving questionnaire • Improving solicitation and sampling • Improving analysis You will be receiving an email surveying the willingness of your organization to participate in this study