Patty Broers Research Coordination Unit Chief

1. Patty Broers Research Coordination Unit Chief

2. Evaluation and Performance Review

3. Points to Ponder • “The most serious mistakes are not being made because of wrong answers. The truly dangerous thing is asking the wrong question.” Peter Drucker • “One of the great mistakes is to judge policies and programs by their intentions than their results.” Milton Friedman • “True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.” Winston Churchill

4. Contract Research Organizational Structure David Lippert Engineer of Materials and Physical Research Amy Schutzbach Engineer of Physical Research Research Coordination Unit Patty Broers - Research Coordination Manager Tom Bukowski – Graduate Public Service Intern (GPSI)

5. ICT Administrative Structure

6. Intergovernmental Agreement Requirements • Quarterly reports • IDOT approval of subcontractors • IDOT approval of work plans/budgets before work can begin • Interim deliverables and final report • Semi-annual evaluations – TRP/PI

7. Research Problem Statement Semi-annual solicitation

8. Key Element: IDOT Sponsor/Champion

9. Key Element: Clearly Identify Benefit to IDOT

10. IDOT’s ICT Executive Committee Organizational Structure Executive Committee Members – Directors of Offices/Divisions within IDOT, one member from FHWA, ICT Director BMPR Environment Public/Intermodal Transportation Structures and Bridges Pavement & Materials Traffic/Roadside Safety Planning Construction TAG’s as appropriate

11. Technical Advisory Group (TAG) Activities Determine research needs: • Review and/or create problem statements • Prioritize/recommend to executive committee • Appoints technical review panel to oversee funded research

12. Technical Research Panel (TRP) Activities • Evaluate RFP submittals and select researcher • Provide technical expertise • Develop an implementation plan • Communication • Review quarterly progress reports • Evaluation – semi-annual and close-out • Champion implementation of research findings when possible

13. Implementation Plan • Required for each project • Plan should identify: • Needs and benefits • Primary panel members to oversee research • Ad hoc members as project progresses needed to promote implementation • Interim deliverables • Necessity for upper management close-out

14. Developing Evaluation Tools

15. RFP Evaluation • Oversight panel evaluates: • Research team • Proposal approach • Strengths/Weaknesses • Equipment/Facilities

16. Project Dashboard Connected to web-based quarterly reports http://ict.illinois.edu/IDOTprojects.asp

17. Quarterly Report • Web-based • Automatic notifications • TRP chair comments

18. Principal Investigator Evaluation • Deliverables, communication, and timeliness • “Needs Improvement” justification • PIs similarly evaluate TRP

19. Principal Investigator Evaluation of Oversight Panel • Panel Evaluation: • Good communication • Clear objectives • Timely report review

20. Close-out Evaluation • Series of questions by e-mail • Rate success of research (1–low to 5–high) • Explain results expected and deliverables received • If applicable, describe what was not accomplished • Indicate implementation actions taken • Improvements that could be made in the future regarding similar research

21. Administrative Performance Evaluation • Evaluate: • Timeliness • Responsiveness • Communication • Problem resolution

22. Administrative Meetings • Monthly meetings • Minutes that include action items • Key issues: • Project meeting issues • Discuss semi-annual evaluations • Budget concerns • Investigator selection

23. The Lincoln Family Abraham Lincoln Presidential Library and Museum Illinois Old Capitol Building