70 likes | 93 Views
Evaluation Tools & Approaches for Engaging Stakeholders: The AREA Approach. Presentation at the Canadian Evaluation Society Dr. Mariam Manley, Principal Evaluator Mid-Continent Research for Education & Learning Mmanley@mcrel.org June 1, 2003.
E N D
Evaluation Tools & Approaches for Engaging Stakeholders: The AREA Approach Presentation at the Canadian Evaluation Society Dr. Mariam Manley, Principal Evaluator Mid-Continent Research for Education & Learning Mmanley@mcrel.org June 1, 2003
What is the Advanced Reading Enhancement Approach (AREA)? AREA is a statewide program designed to help ensure that, by the third grade, all South Dakota students are reading and writing at grade level. The AREA project trains and mentors teachers over an entire school year and consists of the following elements: Ø45 hours of direct instruction to teachersØ18-22 hours of administrator trainingØOngoing mentoring & coaching ØProvision of resources & materialsØStudent Assessment
External, third party Objectivity Validity Reliability Time Consensus (yikes!) Friend/Foe Useful Meaningful Ownership Cost Effective Educational Stakeholder Involvement … Do I REALLY want to do this?!
Methods of Stakeholder InvolvementØLogic Model and evaluation designØInstrument developmentØData collectionØDesign & management of data collection systemØFeedback mechanismsØReporting
DEVELOPMENT OF A PROGRAM LOGIC MODELResources/Inputs Activities Short-term Intermediate Long-term Impact External conditions influencing successØUnderstanding & “Big Picture”ØIdentification of outcomesØLinks between activities and outcomesØCommunicationØFormativeØIdentify evaluation questions and sources of existing informationØReporting
What types of information were collected as part of the AREA evaluation?ØDocumentation and ongoing monitoring of program implementationØPre and post teacher & administrator surveysØClassroom observationsØStudent assessment data ØSite visits & interviews
INSTRUCTIONS FOR USEWhat is the sequence of events in using these instruments?1.Arrange the observation session(s) with the teacher2.Ask a pre-observation question (2 minutes or so)3.Observe the instructional area to be focused upon4.Conduct a post-observation interview (approximately 15 minutes)5.Complete the observation form6.Enter the rating form into the web-based data collection system