1 / 10

Using Accountability to Enact Change in a Program

Using Accountability to Enact Change in a Program. Presentation to the Canadian Evaluation Society/Société Canadienne D’ Évaluation Vancouver, B.C. June 4, 2003. Zoe A. Barley Vice President for Research and Evaluation Mid-continent Research for Education and Learning zbarley@mcrel.org.

osbornea
Download Presentation

Using Accountability to Enact Change in a Program

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Accountability to Enact Change in a Program Presentation to the Canadian Evaluation Society/Société Canadienne D’ Évaluation Vancouver, B.C. June 4, 2003

  2. Zoe A. Barley Vice President for Research and Evaluation Mid-continent Research for Education and Learning zbarley@mcrel.org

  3. Three Levels of Program • The Government (U.S.) Performance and Results Act of 1993 (GPRA) created a framework for ongoing monitoring of the quality and value of government services. • The Regional Education Laboratories (REL) program of the U.S. Department of Education funds ten labs each serving a different geographic region of the U.S. and has its own set of GPRA indicators. • Each individual laboratory and its regional constituents including state departments of education and PK – 16 institutions of education as well as the publics they serve report on the indicators which are then aggregated to the Lab Program level.

  4. 2003 Goal 8 - U.S. ED To promote knowledge-based educational improvement to help all students meet high standards through development, applied research, dissemination, and technical assistance. 2004 Goal 8 - U.S. ED Support evidence-based educational improvement through high-quality, relevant, useful, applied research, development, technical assistance, and dissemination. Changes in the Program

  5. 2003 Performance Indicators An increasing number of sites will be engaged in development. After 3 years on-site sites will show increases in student achievement. Receipt of products and services will increase annually. 90% of clients will report products and services to be of high quality 2004 Performance Indicators Percentage of new evidence-based products and designs: Deemed to be of high-quality by an independent panel. Deemed to be of high relevance to policy or practice by a panel Percentage of all products and services deemed to be of high usefulness to policy or practice by target audiences. Changes in Basis for Accountability

  6. 2003 Data Collection Labs complete forms about sites and determine site outcomes including student achievement. Labs determine and report if outcomes obtained Labs survey clients and report on product quality 2004 Data Collection Labs submit list of evidence based products to IES IES draws random sample for review IES appoints national independent panel of reviewers IES obtains rating of quality and relevance form panel. Changes in Accountability Mechanisms

  7. What is/should be the return on the public investment to which labs are accountable? • State level departments of education receive staff training, technical expertise, informational material, networking, etc. • Intermediate level educational agencies receive technical expertise and training of staff to be trainers of local educators. • Schools and districts serve as development and demonstration sites for new programs and then receive new program materials. • For each of the levels, applied research, training materials, informational products, guides, manuals, etc are regularly provided.

  8. Previous Indicators: Problem – lacked focus on research, on impact, on building capacity of other agency’s staff and on policy work. Solutions – the struggle to improve the indicators New indicators: Problem – entire focus is on products and within that evidence-based products. Solutions – negotiate a better balance with ED so accountability better serves clients. CES #1 What accountability mechanisms are, or should be, in place to ensure that pubic expenditures are appropriately invested to achieve desired outcomes?

  9. Previous indicators Focus on REL System, work is to standardize methods, quality – increase openness among agencies. New indicators Focus needs to be on all three levels, to be accountable to new goal but also to public and contracts in place, must work with ED to negotiate system that serves all. CES #2 How does evaluation help to foster a culture of accountability within public organizations?

  10. CES #4 How can we adapt evaluation practice to help empower the public to hold government accountable for program outcomes? • The change in program goals has actually required us to engage the publics – in this case the commissioners of education in our states and our Congressional delegations – to make them aware of the changes and the implications they have for the services they receive.

More Related