AEA’s Evaluation Policy Initiative

1. AEA’s Evaluation Policy Initiative William Trochim, 2008 President of the American Evaluation Association George Grob, Consultant to the AEA Evaluation Policy Task Force

2. What Is Evaluation Policy? • Any rule or principle that a group or organization uses to guide its decisions and actions regarding evaluation. Can include… • How evaluation is defined • When it gets employed • On what programs • What kind of methods are used • Who does it • What resources are available for it • How it is used • And much more

3. Why Is It Important? Evaluation policy affects • The scope and nature of evaluations that are performed • The effectiveness and efficiency of public programs which are or are not evaluated • The quality of services and benefits that beneficiaries and the public at large receive • Our professional practice

4. Some Examples • Government Performance and Results Act (GPRA) • Program Assessment Rating Tool (PART) and ExpectMore.gov • HHS one percent evaluation set-aside for many health and human services programs • Federal education evaluation requirements for • High risk testing • Randomized experimental designs for evaluating program effectiveness

5. What Can AEA Do About It? • Many evaluation policy decisions are made without the systematic input and advice of evaluators • AEA is well positioned to be the source of such advice • AEA Board creates the Evaluation Policy Task Force (EPTF) • Two year charge • Reports to the AEA Executive Committee

6. EPTF Goal To assist AEA in developing an ongoing capability to influence evaluation polices that are critically import to the practice of evaluation

7. Evaluation Policy Task Force Members • Eleanor Chelimsky • Leslie Cooksy • Katherine Dawes • Patrick Grasso • Susan Kistler • Mel Mark • Stephanie Shipman • William Trochim, Chair • George Grob (Consultant)

8. How Does the EPTF Work? • Consultative Campaign • Public Presence Initiative • Proactive and Reactive Activities

9. Consultative Campaign • Scan emerging Federal legislative, regulatory, and administrative policies to identify targets of opportunity • Brief congressional and executive staff about the value of evaluation, effective evaluation practices, and using evaluation results for accountability and program improvement • Network with contacts both within and outside of government that can assist AEA in influencing evaluation policy • Respond to requests for input on policy formulation

10. Public Presence Campaign • Consultations. Consult with evaluators and others experienced in policy development • Materials. Develop collateral materials, e.g. white paper, brochures, AEA Guiding Principles • Website. Advise on AEA website enhancements for policy influence

11. Proactive and Reactive Activities • Target a few specific legislative and executive policies • Focus on selected substantive areas • Be appropriately opportunistic

12. Responsibilities and Participation • Broad input from and consultation with AEA Members • EPTF will guide the initiative and make policy recommendations to the AEA Executive Committee and Board • The Consultant will assist and support the EPTF and be the lead on implementation

13. Next Steps • Identify highly influential evaluations • Learn from • Experienced Evaluators • Professional Organizations • Establish “Policy Scan” function • Prepare briefing kits • Consult with experienced federal evaluators • Meet the policy makers • Congressional • Executive Branch • Identify evaluation policy opportunities

14. Discussion Questions • What are the key policy opportunities that we should consider pursuing? • What are the best ways to engage evaluators, policy makers, and program managers?

15. Staying In Touch • Email: Evaluationpolicy@eval.org • Website: http://www.eval.org/EPTF.asp • Discussion Group: http://www.eval.org/EPTF.signup.discussion.asp