Office of Governmentwide Policy Center for Policy Evaluation (CPE)

1. Office of Governmentwide PolicyCenter for Policy Evaluation (CPE) February 4, 2009

2. Part 1: Background Methodology Changes for 2009 Evaluation Criteria and Scoring 2008 Results 2009 Schedule Part 2: Website Policy Review Tool Sections Submission Process System flow Emails Reports Additional Information Agenda

3. Background • Outgrowth of OMB PART evaluation • Facilitate government-wide reform: • Measuring and improving policy • Adopting best practices • Implementing innovative tools

4. Interagency Committees/Councils Federal Mail Executive Council Property Management Executive Council Motor Vehicle Executive Council Interagency Mail Policy Council Interagency Council on Property Management FedFleet Policy Council Executive Travel Steering Committee Interagency Committee for Aviation Policy Executive Relocation Steering Committee Interagency Travel Management Committee E-FAS Executive Steering Committee Transportation Management Policy Council

5. Overall Process Agency writes Standard Operating Procedures and internal guidance OGP discusses policies w/ Committees & OMB OGP establishes or changes policy Agency implements or supplements GSA policy Continuous Improvement Policy Evaluation Role Agency verifies policy is followed in the agency OGP identifies strengths, weaknesses, opportunities and threats OGP evaluates agency-provided information Agency reports to OGP OGP - GSA’s Office of Governmentwide Policy

6. Roles and Responsibilities Travel Relocation Motor Vehicles Personal Property Aircraft Mail Transportation OGP DHS USSS USCG FEMA TSA CBP CIS ICE

7. CPE’s Approach • Conduct agency policy evaluations via Internet • Provide initial/final assessment of mandates • Evaluate Best Practices • Capture OGP feedback from customers • Final written evaluation and score • Evaluate and post government-wide data

8. Linking: Policies, Goals & Measures • Mandates & Best Practices linked to: • Policy, guidance, industry best practices • Suggested performance measures • Strategic goals: • Efficiency/Effectiveness • Accountability • Safety & Environment

9. Link Example

10. Changes for 2009 • Modified evaluation criteria and scoring • Fewer scored questions • Best Practices not scored by agency • Additional time built into schedule • Specific question regarding familiarity with mandate and greatest challenge to adhere to mandate • Required feedback to OGP/MT and CPE • Password reset functionality • Additional reports

11. Evaluation Criteria • Written Policy: Address the mandate, and provide evidence of current policy implementation • Awareness: Evidence of communicating with affected agency personnel about the mandate, including policy or implementation changes • Verification: Check adherence with applicable documented performance measures More details available at www.gsa.gov/cpescoring

12. Scoring Matrix

13. 2008 Results • Assessed 7 policy areas - 23 executive agencies • 87% participation • Focus Group: 100% planned to participate again • Final assessments provided to agencies • Published the 2008 Policy Evaluation Report (See 2008 Report)

14. All Policy Areas

15. All Policy Areas: Interim to Final Evaluation 15

16. Key Findings • OGP provides good advice, guidance and recommendations to agencies prior to finalizing policies • Agencies have had some success in adhering to policies; however implementation rates are lower than ideal • Agencies struggle to identify, adopt, and implement best practices • OGP should develop tools for agencies to report required information, and make those reports available on its website.

17. Strengths Collaboration/policy development • Strong relationship between GSA and executive agencies through interagency councils/committees Evaluation criteria • The policy evaluation criteria is a strong model for agency policy staff and serves as a catalyst for making changes Adherence to regulations supports agency missions • 75% of stakeholders feel that participating in evaluation increased agency's attention to policy area(s).

18. Weaknesses Policy implementation • Agencies struggle to implement some policies due to lack of resources, other priorities, or adherence is planned at a later date. GSA's mechanisms for collecting agency information • Internet tools are needed to collect required information for agency annual reports. Dissemination of information • GSA needs to develop mechanisms to disseminate information more quickly and widely. Best practices • Agencies struggle to adopt and implement best practices.

19. Opportunities Transparency of program information • GSA should improve its feedback to agencies and make annual government-wide reports available on its website. Web-based reporting tools • GSA should develop tools for agencies to provide standardized annual reports. Increase training and policy awareness • GSA should develop mechanisms to monitor the Policy Evaluation program, and expand its efforts to provide training and education to executive agencies.

20. Threats Program-specific policy implementation • The failure to implement government-wide mandates poses a potential threat to the effective and efficient management of government-wide programs.

21. Independent Survey Feedback • GSA needs more focus & involvement with agencies • Additional training and more personalized interaction • PRT participation increased attention to policy area • Benefit worth effort • Criteria made sense

22. Actions taken… Implement web-based tools for required annual reports • New web tools with reports published on OGP website Agency adoption of Vehicle Management Information Systems • Motor Vehicle Executive Council and OGP are evaluating the feasibility of using existing systems, like GSA’s FAS or DOE, to close this gap Develop Aviation/Fleet Capital Asset Planning • OGP established an Interagency Integrated Project Team to develop standard capital planning processes, procedures and tools.

23. 2009 CPE Schedule

24. CPE Web (www.gsa.gov/cpe)

25. Components of PRT 4 Parts: Help, Forms, Reports, Tools • Help: Detailed instructions, FAQs • Data Collection Forms (DCF): Data input • Reports: Output • Tools: Define users & approvers https://gsa.inl.gov/prt

26. Data Collection Forms (DCFs) • Mandates • GSA policy, your agency, IG, OMB, GAO, PART • Best Practices • MT and CPE Feedback • Agency Respondents and Approvers provide feedback to both MT and CPE • Questions are not scored • Help identify what works & needs improvement

27. Mandate & Best Practice Layout Four sections: • Mandate/Best Practice statement, adherence questions, familiarity and challenges • GSA interim response & assessment • Agency follow-up reply (if needed) • GSA final response

28. 7 Challenges to Adherence • Other priorities precluded compliance 2. Lack of resources - funding 3. Lack of resources - personnel 4. Compliance planned for later date 5. Unaware of requirement 6. Received GSA waiver 7. Other

29. Submission Process • AR completes four sections: • Mandates, Best Practices, MT and CPE feedback • AA completes MT & CPE feedback & submits to CPE • CPE provides agency interim scores • Agency adds info and resubmits • CPE provides final agency evaluation scores for Mandates, government wide scores for Best Practice questions, and analyzes MT and CPE Feedback

30. PRT Process & Roles/Responsibilities Key GSA/Center for Policy Evaluation (GSA/CPE) Task Agency Respondent (AR) Task Agency Approver (AA) Task PRT opens Agency Respondent (AR) answers questions • Agency Respondent (AR): • Complete data responses • Submit to Agency Approvers (AA) • Revise as needed & resubmit to AA AR completes Initial GSA Feedback section AR submits to Agency Approver (AA) for review Yes AA reviews initial responses Changesrequired? No AA completes Initial GSA Feedback section AA submits initial DCF to CPE CPE provides initial comments and scores *Process repeats for final evaluation

31. PRT Process & Roles/Responsibilities Key GSA/Center for Policy Evaluation (GSA/CPE) Task Agency Respondent (AR) Task Agency Approver (AA) Task PRT opens Agency Respondent (AR) answers questions • Agency Approvers (AA): • Review AU submissions • Approve or disapprove w/ comments • Submit to GSA AR completes Initial GSA Feedback section AR submits to Agency Approver (AA) for review Yes AA reviews initial responses Changesrequired? No AA completes Initial GSA Feedback section AA submits initial DCF to CPE GSA/CPE provides initial comments and scores *Process repeats for final evaluation

32. PRT Process & Roles/Responsibilities Key GSA/Center for Policy Evaluation (GSA/CPE) Task Agency Respondent (AR) Task Agency Approver (AA) Task PRT opens Agency Respondent (AR) answers questions GSA/CPE: Evaluate responses, identify SWOTs, provide agency, government-wide and MT evaluation feedback. AR completes Initial GSA Feedback section AR submits to Agency Approver (AA) for review Yes AA reviews initial responses Changesrequired? No AA completes Initial GSA Feedback section AA submits initial DCF to CPE GSA/CPE provides initial comments and scores *Process repeats for final evaluation

33. Login to PRT

34. What’s New Screen

35. DCF Selection for Program Areas

36. Navigation Screen

37. Mandate Question: Data Input Familiarity: Unfamiliar to expert Best Practice questions have same format

38. Data Input: Attachments Best Practice questions have same format

39. GSA Feedback

40. Send for Agency Approval

41. Awaiting Initial Agency Review

42. Status for Agency Approver Agency Approver sees assigned policy areas and status 43

43. AA Reviews & Provides MT Feedback Agency Approver OGP/MT Feedback

44. Comments from Agency Approver AA can’t change info or comment on individual questions. If disapprove, AR must correct and resubmit to AA.

45. AR Resubmits to AA

46. Agency Approver Submits

47. Awaiting GSA Review

48. CPE Review-Mandates Best Practice questions have same format

49. Emails from the PRT • New users or passwords reset • DCF status changes • System messages

50. Reports & Analytical Tools • Quick Reports show performance by: • MT & CPE Feedback Summary • Performance by Program Area & Strategic Goal • Performance Comparison by Program Area & Strategic Goal, Across all areas • Performance by Question • Adoption Challenges • Performance Summary by Question • Create or use saved queries (available later)