320 likes | 385 Views
FRAME what is to be evaluated. Patricia Rogers, RMIT University, Melbourne. The Rainbow Framework. Why do we need to start with a clear definition?. Source: Hobbies on a Budget / Flickr. Why do we need to start with a clear definition?. Source: Hobbies on a Budget / Flickr.
E N D
FRAME what is to be evaluated • Patricia Rogers, RMIT University, Melbourne Part 3 of 8 AEA Coffee Break Webinars 2013
Why do we need to start with a clear definition? Source: Hobbies on a Budget / Flickr
Why do we need to start with a clear definition? Source: Hobbies on a Budget / Flickr
Why do we need to start with a clear definition? Source: Hobbies on a Budget / Flickr
Why do we need to start with a clear definition? Source: Hobbies on a Budget / Flickr
1. Identify primary intended users
Families Possible primary intended users Teachers Principal School Council Education Department Other schools Image source: IN 157s01 – Curt Carnemark/ World Bank
2. Decide purpose(s)(intended uses)
Formative – improve it Summative – continue or stop it Lobby and advocate Broader evidence base Purposes(intended uses) Image source: CK-CO138 - Charlotte Kesl / World Bank
Provide voice Build trust and legitimacy Purposes(intended uses) Accountability
Interconnection between intended users and intended uses FRAME
3. Specify the key evaluation questions
Descriptive: How many children attend? What learning tools are used? Has learning improved? Key evaluation questions Image source: ML030S09 - Curt Carnemark / World Bank
Causal: Has the program contributed to improved learning? Key evaluation questions
Synthesis: Has the program been a success? Is it Value For Money compared to alternatives? Key evaluation questions
Action: How can the program be improved? Should it continue? Key evaluation questions
Options for answering different types of questions MANAGE DEFINE FRAME DESCRIBE Descriptive Questions- What were the activities, changes, context? UNDERSTAND CAUSES Causal questions –What caused orcontributed to the identified changes? SYNTHESIZE Synthesis questions –Overall was it good? Value for money? REPORT & SUPPORT USE Action questions- What should we do?
4. Determine what success looks like
Processes: Are students fairly treated? What does success look like? Image source: LQ-CN7499 World Bank
Outcomes: Do students learn to read? What does success look like?
Distribution of costs and benefits: Who benefits from the program? Whose needs are met? Who is disadvantaged by it? What does success look like?
Criteria and standards: Reading levels – better than before, or better than national average, or national benchmark? What does success look like?
You can read about this case here: http://betterevaluation.org/blog/outcome_harvesting_bionet
Further resources http://betterevaluation.org/plan/frame facebook.com/betterevaluation @bettereval Patricia Rogers RMIT University patricia.rogers@rmit.edu.au