DoD Acquisition Best Practices Clearinghouse Content Advisory Group (CAG) 12 April 2006

1. DoD Acquisition Best Practices ClearinghouseContent Advisory Group (CAG)12 April 2006

2. Meeting Purpose • Finalize the CAG charter • Review planned content evolution schedule • Current content activities • Planned content activities • Existing content status • Identify pilot working groups • Establish goals • Define the approach Tasking from 14 Sep 05 OSD Briefing: Complete and demonstrate the practice and evidence information handling process such that there is clear data to indicate the “validation and vetting” system can be executed with available and affordable resources in DAU, OSD, Services, Agencies and Industry.

3. Agenda • Introduction and BPCh Program Update [Mike Lambert] • CAG Meeting Purpose/Agenda • CAG Charter and Comments • Other Program Activities • BPCh Content Update [Forrest Shull] • Content Schedule • Risk Management • Earned Value Management • Other Content Areas • Lessons Learned • BPCh Information Handling Process [Michele Shaw / Mike Lambert] • Current Roles and Activities • Scenarios List • Recommended Subgroups • Scenarios to Pilot • CAG Wrap Up [Mike Lambert] • Next Steps / Action Items / Next Meeting

4. CAG Charter Summary • A small group of people familiar with • SE/SA practices that are useful / recommended • Needs / “pain points” of likely users • …that can help improve BPCh: • Define, manage, and track content evolution activities to ensure that they are consistent across the BPCh program goals. • Lead efforts to establish efficient and effective information handling processes for populating content within the BPCh. • Establish and build relationships to support the growth and continued evolution of BPCh content. • Provide an action-oriented collaborative forum for improving content gathering, processing with the overall goal of promoting the increased use of BPCh.

5. Round Table • What have you done for BPCh since we last met? • Suggested an evidence source • Presented BPCh to an interested party • Identified a potential practice owner • Participated on working group • Sent the BPCh link to a potential user • Added content (lead, war story, evidence, practice summary etc) • Submitted BPCh process improvements

6. BPCh Program Update

7. Since we last met… • Recent/Current Activities • $75K each received from DS, ARA & OSD(NII), added to $20K from DAU I.T. brings total BPCh FY06 budget to $245K • Bug tracking system established for current BPCh prototype and issues are being reported/solved (per schedule) • Work on risk mitigation plan to begin in earnest in May (per schedule) • Met w/ DAU taxonomy PM to discuss integration of developing taxonomy into BPCh • “Bigger picture”: working w/ DAU KM team on how best to integrate BPCh & other KM systems into an Acq Knowledge Management System (AKMS), which is a foundational part of DAU’s evolving Engaged Learner Architecture (ELA)

8. Elements of the AT&L Knowledge Management System (AKMS) Central Repository for AT&L Policy and Reference Materials • Also serves as the home for knowledge gateways like: • Defense Acquisition Guidebook • AT&L Integrated Framework Chart (IFC) • Ask A Professor DoD & Industry Best Practices • Will stand alone as a best practices resource • Will also provide content for CoPs to allow for additional collaboration/input on best practices • Will be included in the enterprise search index/results Enterprise Search System • Stand-alone search and discovery for AT&L workforce • Integrated search for AKSS • Searches open areas of ACC • Integrated search for DAU Homepage • Integrated search for DAU Intranet Integrated System of Systems Collaborative Tool for the AT&L Community Where the Workforce Contributes Knowledge and Interacts to Share “Know-How” • Provides a nest of collaborative tools: • Communities of Practice/Interest • Special Interest Areas • Limited Access Workspaces • DAU Course Spaces • Workflow Learning Tools • IFC Templates

9. Program Concerns or Issues • Planned FY06 activities were based on committed funds of $435K; confirmed and received FY06 funding stands at $245K • Curtailed activities (or postponed to FY07) may include: • Pursuit of/formalizing PPN partnerships • Reduced risk mitigation efforts • Prototype testing & evaluation master plan • Engaging DAU faculty/staff in the practice/evidence information handling processes and to become practice “owners”

10. The BPCh Practice Providers NetworkResponsible Content Sources - “Trusted Sites” Offices/individuals in OSD, Services, Agencies, Industry that are responsible for capturing “local” best practices and lessons learned for their workforce: • OSD • Defense Acquisition Policy Working Group (DAPWG) – Skip Hawthorne • AT&L Functional Advisors for workforce career management – Various OSD Leadership • DAU Curriculum Development Support Center – Bill Erie/Dr. Ainsley/CDs/PDs • Services • Air Force • Acquisition Center of Excellence (ACE) – Terry Balvin • Air Force CIO/Air Force Knowledge Now (AFKN) – Randy Adkins/Harry Pape • Army • Acquisition Lessons Learned Management System (ALLMS) – Major Ellison • Army CIO/Chief Knowledge Officer – Gary Winkler • Marine Corps • MARCORPSYSCOM Tiger Portal – Barry Dillon/Dick Bates • Navy • DON Acquisition One Source – Christine Stelloh-Garner/Page Glennie • DON CIO/Chief Knowledge Officer – Jim Knox • Agencies • DCAA, DCMA, DLA, MDA • Industry – Lenn Vincent, Radm USN, Retired, DAU Industry Chair • Aerospace Industries Association (AIA) – John Douglas • Boeing – Alan Brown • LMCO – Paul Robitaille • Northrop/Grumman – Curt Newill • Raytheon – Mike Sharpe • Data and Analysis Center for Software (DACS) • IEEE (Technical Council for Software Engineering) • International Council on Systems Engineering (INCOSE) – Paul Robitaille/Dr. John Snoderly • National Defense Industrial Association and Related Special Interest Groups – Bob Rassa • Software Engineering Institute (SEI) of Carnegie Mellon • Systems and Software Consortium (SSCI) – Dr. Richard Turner

11. BPCh Content Update

12. When we last met… • Reviewed the System Engineering Plan • Agreed that the SEP would drive the capture of content • Brainstormed practices / categories • Sticky votes on listed practices / categories • Identified “Top 4” Practices / Categories • Risk Management (5) • Earned Value Management (5) • Software Assurance (4) • Spiral Development (4) • Suggested Practice / Category Owners

13. Definitions • Lead • Unprocessed data coming into the BPCh • Experience • Lessons Learned • Stories • Practice • Not from the Preferred Provider Network • War Story • Lead in narrative form possibly associated with a practice • Evidence • External data (lead or PPN input) that has been processed and entered into the BPCh • One-to-many relationship to practices • Practice • Repeatable • 0 to many evidences • Trusted Published Source • A source or site identified by the content manager from which content (practices, evidence, stories) can be expedited

14. BPCh User or Information Provider Start 1.1 Enter Leads 1.2 Process Leads 1.3 Enter Evidence Practice Processor or Information Handler 1.3 Enter Evidence Practice Provider Network Leads andStories Evidence BPCh 1.5 Process Stories 1.4 Review Evidence Practice Owner (Expert) Info Handler Process Pilot • Since February: Piloted Info Handler activities • Developing guidelines & templates Next phase: Pilot mechanisms for increasing leads and stories from live user community

15. Content Evolution Schedule Feb Mar Apr May Jun • Since February CAG: EVM & Risk Management • Surveyed the literature • Mined ACC / COP CAG members “market” BPCh: Find experts & content sources CAG members “market” BPCh: Find experts & content sources CAG statusmtg CAG statusmtg CAG kickoffmtg Content developers (FCMD & others) add evidence from pubs, interviews, trust sites Content developers (FCMD) add evidence from pubs, interviews EVM & RM Existing sites are linked and/or mined for content Content owners review material Content developers (FCMD) add evidence from pubs, interviews IA & Spir Existing sites are linked and/or mined for content

16. EVM Practices: leads Responsible; Dave Bachman (?) resources published evidence war stories interviews trusted sites Content Growth Overview evidence summary vetted

17. Risk Management Practices: leads Responsible; Bill Bahnmaier (?) resources published evidence war stories interviews trusted sites Content Growth Overview evidence summary vetted

18. Content Concerns or Issues • Identified a need for tool support, esp. related to work flows • Working this with CSC development team • When to involve experts/practice owners? • When do we have enough evidence? • Sufficient attribute or context coverage driven initially by the CAG and ultimately by the users • How to get more leads/stories/ideas from users? • Topic owners vs. practice owners

19. Next Steps • EVM & Risk Mgmt: • Pilot interviews • Meet with Practice Owners & establish responsibilities • Info assurance & Spiral Development: • Begin with content mining • Kickoff meetings with POs

20. Information HandlingProcess Roles and Activities

21. BPCh User or Information Provider Start 1.1 Enter Leads 1.2 Process Leads 1.3 Enter Evidence Practice Processor or Information Handler 1.3 Enter Evidence Practice Provider Network Leads andStories Evidence BPCh 1.5 Process Stories 1.4 Review Evidence Practice Owner (Expert) Info Handler: Processing Evidence • Practice Processors: • Enter leads on behalf of end users (elicit the relevant information from e.g. an email, maybe follow-up if info too light) • Turn selected leads into evidence profiles with BPCh • Enter evidence profiles from the literature or other sources

22. BPCh User or Information Provider Start 2.3 Identify New Practices 2.4 Identify Practice Owner Content Manager & CAG 2.5 Enter Approved Practices Start Practice Owner (Expert) 2.1 Enter Feedback 2.2 Evaluate Feedback 2.7 Process Trusted Information 2.6 Enter Candidate Practices Practice Processor or Information Handler Trusted Published Site 2.6 Enter Candidate Practices Practice Provider Network Start Feedback Practices BPCh Info Handler: Identify Practices • Practice Processors: • Mine information from trusted sites and create records in BPCh • May enter feedback on behalf of users • Evaluate feedback to mine ideas for new practices, pass good candidates along to CAG

23. BPCh User or Information Provider Start 1.1 Enter Leads 1.2 Process Leads 1.3 Enter Evidence Practice Processor or Information Handler 1.3 Enter Evidence Practice Provider Network Leads andStories Evidence BPCh 1.5 Process Stories 1.4 Review Evidence Practice Owner (Expert) PPN: Process Evidence • Practice Provider Network: • Trusted members of the community who can enter evidence directly into the database (e.g. KPOs extracting knowledge from CoPs)

24. BPCh User or Information Provider Start 2.3 Identify New Practices 2.4 Identify Practice Owner Content Manager & CAG 2.5 Enter Approved Practices Start Practice Owner (Expert) 2.1 Enter Feedback 2.2 Evaluate Feedback 2.7 Process Trusted Information 2.6 Enter Candidate Practices Practice Processor or Information Handler Trusted Published Site 2.6 Enter Candidate Practices Practice Provider Network Start Feedback Practices BPCh PPN: Identify Practices • Practice Provider Network: • Maintain their own repositories of helpful knowledge and may provide some level of assistance for BPCh to process & link to them • Can submit ideas for practices BPCh ought to include

25. BPCh User or Information Provider Start 1.1 Enter Leads 1.2 Process Leads 1.3 Enter Evidence Practice Processor or Information Handler 1.3 Enter Evidence Practice Provider Network Leads andStories Evidence BPCh 1.5 Process Stories 1.4 Review Evidence Practice Owner (Expert) Practice Owner: Process Evidence • Practice Owners: • Review (sanity-check) evidence entered on their practice • Review relevant “stories” to see which contribute useful info and should be turned into evidence • Suggest practices, evidence, sources

26. BPCh User or Information Provider Start 2.3 Identify New Practices 2.4 Identify Practice Owner Content Manager & CAG 2.5 Enter Approved Practices Start Practice Owner (Expert) 2.1 Enter Feedback 2.2 Evaluate Feedback 2.7 Process Trusted Information 2.6 Enter Candidate Practices Practice Processor or Information Handler Trusted Published Site 2.6 Enter Candidate Practices Practice Provider Network Start Feedback Practices BPCh Practice Owner: Identify Practices • Practice Owner: • Assigned by CAG when a new practice record is created • Responsible for entering links to useful resources, summary (when required), and other information in the practice record • Enter practices without review by the CAG

27. BPCh User or Information Provider Start 2.3 Identify New Practices 2.4 Identify Practice Owner Content Manager & CAG 2.5 Enter Approved Practices Start Practice Owner (Expert) 2.1 Enter Feedback 2.2 Evaluate Feedback 2.7 Process Trusted Information 2.6 Enter Candidate Practices Practice Processor or Information Handler Trusted Published Site 2.6 Enter Candidate Practices Practice Provider Network Start Feedback Practices BPCh CAG: Identify Practices • Content Manager & CAG: • Determine new practices to go after • Identify and support owners of new practices

28. Information HandlingOperationalScenarios

29. Scenarios • Process Evidence • Lead from a BPCh User that is Evidence • Lead from a BPCh User that is a Weak War Story • Lead from a BPCh User that is Strong War Story • Evidence or War Story from PPN (including trusted published site) • Process Feedback and Identify Practices • Candidate Practices from BPCh User • Candidate Practices from PPN • New Practices from Practice Owner • New Practices from CAG • Feedback from a BPCh User • Prepare and Validate Summaries

30. Information Handling Scenario Pilot Approach

31. BPCh FY06 Activity (Identified Funding) • Minimum analytical analysis to prepare for MS B. Basic Analysis of Alternatives & Risk Reduction w/ Prototyping • Update and complete existing AoA & Risk Management Plan to include the existing BPCh platform and the following possible options: • Integration of existing BPCh database & information handling system with new ACC enterprise upgrade (ECCO) • Integration of BPCh with Microsoft SharePoint portal system • Update and complete Concept of Operations (ConOps) • Test and evaluate a BPCh prototype leading to Milestone B • Review and firm up BPCh information handling process • Formalize Best Practice Providers Network partnerships • Leverage DAU student and faculty resources for risk reduction efforts and development of CDD • Develop joint service acquisition reference “taxonomy” • Support Workforce Management BP Clearinghouse initiative

32. Scenarios Working groups? • Process Evidence • Lead from a BPCh User that is Evidence • Lead from a BPCh User that is a Weak War Story • Lead from a BPCh User that is Strong War Story • Evidence or War Story from PPN (including trusted published site) • Process Feedback and Identify Practices • Candidate Practices from BPCh User • Candidate Practices from PPN • New Practices from Practice Owner • New Practices from CAG • Feedback from a BPCh User • Prepare and Validate Summaries

33. Proposed Pilot Approach • Form CAG Working Groups for each Key Scenario • Identify Working Group Leader • Develop working group plan/schedule and coordinate with project manager • Review existing scenario process flows to validate • Do pilot • Implement scenario process flow with content • Coordinate content to use with Content Manager • Collect data • Recommended process flow changes • Time spent on key process flow steps • Develop guidelines to support execution of the process • Provide status to BPCh Project Manager • Prepare and analyze process implementation results • Report results to CAG

34. Process Concerns or Issues • Are sufficient resources available to complete the working group activities? • Are the working groups goals aligned with needs for the upcoming milestone? • What’s the timeframe for completing the working group results?

35. CAG Wrap-Up

36. Exit Criteria / Next Steps • Are the targeted content categories and practices still the correct priority? • Are there other content resources for risk management and EVM that we should go after? • Do we have a list of scenarios that we will pilot? • Do we know the approach for piloting the scenarios? • Have we set a date for next meeting to review the content?

37. Backup Slides

38. CAG Charter Comments

39. Content Growth Overview:EVM Resources Show resources on the BPCh practice page (with links) ??

40. Content Growth Overview:Risk Management Resources

41. March 30 - Dry Run Agenda • Review Proposed CAG Material • Time boxing 9:00 – 1:00 • BPCh Front Page Enhancements • ANSI Presentation • Planning Meetings (Configuration Control Board) • Funding Status • Demonstrate BPCh Backend • Other Topics

42. BPCh Program Current Status • OSD principle officers were briefed on 6 January and the following decisions were established: • Move forward with the BPCh development plan as briefed, using identified FY 06 funding ($435K), to accomplish the following objectives: • All FY06 funding activities highlighted on previous slide, including the assessment of “Managing the Force” Best Practices Clearinghouse initiative for possible integration with the AT&L BPCh • Develop a business case and document requirements on the way to meeting other Milestone B requirements by the end of FY 06. • Designate Mr. Anderson as the Acquisition Executive (AE) who becomes the Milestone Decision Authority for BPCh, until or unless the newly established Business Transformation Agency’s (BTA) AE, MG Carlos “Butch” Pair is identified as AE. • A Memoranda of Agreement (MOA) outlining governance structure and signatories among BPCh sponsors is NOT now required. Mike to keep or update

43. BPCh Program Current Status • Final tasking from OSD principle officers briefing on 6 January: • Compose Memorandum for Record documenting group discussion/decisions; MFR written in mid-January and is currently being coordinated by Col. Anderson for approval/signature among group principles. • BPCh FY06 funding from NII and DS ($75K each) in process of being transferred to DAU; still working receipt of committed funds from ARA and crosscutting study. • Working on draft partnership language and tailored program overview presentation for potential Practice Providers Network participants and POCs • Working w/ DAU Taxonomy PM to determine how we will incorporate taxonomy metadata into BPCh design • Biweekly status meetings to re-commence on 28 Feb Mike to keep or update

44. BPCh Development Schedule MOU to be Signedbefore or at MS-B Start Turnover of PM responsibility to DAU AoA / Risk Reduction Jul 2006 DAU Dev/Testing Oct 2006 NDIA SE Conference Oct 2005 SSTC 21-25 April 2005 SSTC Apr 2006 MS-B “Alpha” prototype version developed and demonstrated Version 1.0FSD begins AoA & Risk Mitigation ongoing; content generation, security and user testing at DAU “Beta” prototype version evolved and demonstrated Primary Lead: CSC Supporting: DAU-IT & FC-MD Primary Lead: FC-MD Supporting: N/G-IT Primary Lead: CSC Supporting: FC-MD Primary Lead: CSC Supporting: DAU-IT & FC-MD IOC Apr 2007 Oct 2007 Apr 2008 Oct 2008 MS-C(V1.0) (V2.0) Acceptance Public access SE & SAM focus Start O&S Industry focus PM & Log focus Industry focus Complete all AT&L Functions Functional Updates (every six months)

45. BPCh demo: http://bpch.dau.mil • Latest front-end, integrated with DAU KS assets

46. Evidence 1 SourceContextResults Evidence 4 SourceContextResults Repository Forms/Data Structures Practice A Evidence 2 SourceContextResults Evidence 3 SourceContextResults

47. Practice A Summary of sourcesSummary of results in context Evidence 1 SourceContextResults Evidence 4 SourceContextResults Evidence 2 SourceContextResults Evidence 3 SourceContextResults Repository Forms/Data Structures

48. Evidence 1 SourceContextResults Evidence 4 SourceContextResults Evidence 2 SourceContextResults Evidence 3 SourceContextResults Repository Forms/Data Structures Practice A Summary of sourcesSummary of results in contextVetted as of March 2005

49. Evidence 1 SourceContextResults Evidence 4 SourceContextResults Evidence 5 SourceContextResults Evidence 2 SourceContextResults Evidence 3 SourceContextResults Repository Forms/Data Structures Practice A Summary of sourcesSummary of results in contextVetted as of March 2005

50. BPCh demo: http://bpch.dau.mil • Latest back-end, to manage practice information