NCHRP 3-94: Systems Operations and Management Guide

1. NCHRP 3-94: Systems Operations and Management Guide Steve Lockwood, Phil Tarnoff, John Conrad and Rich Margiotta Presentation to AASHTO SSOM Manchester, NH June 16, 2009

2. Agenda • Introduction and Approach • The Black Box – How it works • Example of Pilot Application • Web Product Concept • Sample Strategies for Review

3. Introduction • Objective: Mainstreaming SO&M as formal SDOT program • Guidance for SDOT managers • Develop existing material into an accessible and user-friendly product • Web-based approach

4. Benefits of Web-Based Approach • Avoids lengthy paper documents with big charts • Relationships among elements built in • Custom tailored to user • Users self-evaluate SDOT state of play • Increasing levels of detail displayed on automated basis • Hyperlinks to supporting documents

5. Approach • Covers interrelated elements critical to improving SO&M performance • Process and scope • Institutional elements (including organization)

6. Operations Capability Maturity Levels:Process Dimension Levels and Supporting Institutional Dimension Levels Goal for the future A few Leaders Integrated Transitioning Agencies (most) Level 3 Managed Fully coordinated, performance-driven Level 2 Ad Hoc Level 1 Processes fully documented & staff trained Support Arch. Ad hoc operation. Relationships not coordinated Support Arch. Support Arch.

7. 1. Business processes Scoping/Planning Program development Programming and Budgeting Procurement Operations 2. Systems and Technology Regional architectures Project systems engineering (design) process Standards/interoperability Testing and validation Maintenance 3. Performance Measures definition Data acquisition management Measures utilization The Process Dimension

8. 1. Mission Mission commitment Operations Culture SDOT authorities (laws) Continuous improvement acceptance (culture) 2. Leadership In-reach Outreach (policy-makers, stakeholders) 3. Organization Status/authority (equivalence) Unit relationships (consolidation) Authority/responsibility allocation 4. Staffing Core capacities definition/filled Technical capacities (staff development) Career path (incentives), Recruitment and retention 5. Resources Funding sources Budgeting/resource allocation 6. Partnerships Interagency (cooperation, co-training) Intergovernmental (cooperation, coordination) Participation in MPO activities PPP (rationalized outsourcing) The Institutional Dimension

9. Agenda • Introduction and Approach • The Black Box – How it works • Example of Pilot Application • Web Product Concept • Sample Strategies for Review

10. Framework: The Capability Maturity Model • Performance-critical elements identified (process, institutional) • Self-evaluation based by user type (administrator, project manager, program manager) • Presumes continuous improvement in critical elements • Strategies provided for levels of user to get to next level • (different strategies for different user types and their levels) • Each strategy has guidance including , examples, reference • Criteria for levels, strategies, guidance are in an invisible data base

11. Agenda • Introduction and Approach • The Black Box – How it works • Example of Pilot Application • Web Product Concept • Sample Strategies for Review

12. A recent "manual" application of the guide logic • State of Old Hampshire • Medium size, some SO&M programs • Some Policy-based strategy applications • Start-up Performance measurement context • Has an aggressive SO&M vision • Example below:

13. Program Assessment: Strengths and Weaknesses • Overall assessment in terms of maturity levels (for selected critical process elements) • Scope • Staff Management • Customer Service • Business Processes • Technology and Systems • Performance Measurement

14. Ranking and rationale • Action Plan -- starting with lowest level • Strategies to move up next level • Investment implications

18. Agenda • Introduction and Approach • The Black Box – How it works • Example of Pilot Application • Web Product Concept • Sample Strategies for Review

19. Step 1 – User Role Self-identification • Indicate User’s position (check one): • Top management (CO or District) • Program manager (CO or District) • Project manager (CO or district)

20. Evaluation goes element-by-element

21. Step 2: User self-evaluation of current level of capability in all key elements Ex: Systems and Technology – Regional Architecture • Need for architectural standardization among state and regional systems are ignored • Regional architecture developed and subjected to periodic reviews • System implementations recognize need for future upgrades for increased functionality • Regional architecture recognized as a dynamic tool requiring continuing review and updates

22. Note: User will repeat step for other 7 elements later • Answer similar questions regarding: • PROCESS DIMENSION • Business processes (including scope) • Systems and Technology • Performance • INSTITUTIONAL DIMENSION • Mission • Leadership • Organization / Staffing • Resources • Partnerships

23. Step 3: Display of user’s level for each element at lowest level

24. Step 4 System Determines Appropriate Strategy

25. Step 5 – Guidance for Appropriate level transition displayed

26. (Note: Other layers are stored by level and position)

27. (Note: Other layers are stored by level and position)

28. Step 6: Store and continue with other elements • System shows accounting matrix for components checked and next component • User checks “next” • System goes to next most critical element

29. Step 7 to N: system repeats steps 3-6 • Step N + 1: completion high level summary display of change management strategies • “You were ranked as xxxxxxxxx, and we recommended the following” • Print-out functionality

30. Agenda • Introduction and Approach • The Black Box – How it works • Example of Pilot Application • Web Product Concept • Other Sample Strategies for Review

31. Performance Measurement : General Strategies

32. Performance: Level 1 to Level 2

33. Performance: Level 2 to Level 3

34. Performance: Level 3 to Level 4

35. Performance: Level 3 to Level 4 (continued)

36. Step 3: Display of user’s levels and indication of lowest level

37. Step 4: Display of Strategies to get to next level for lowest level (“critical” component)

38. Step 5: Detailed Guidance for moving selected component to next level