380 likes | 549 Views
Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement. Margaret Heritage CRESST/UCLA CBSE & ACER International Educational Conference on Assessment 25 th January, 2011 Delhi, India. Overview. Recent U. S. Developments A System of Quality Assessments
E N D
Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement Margaret Heritage CRESST/UCLA CBSE & ACER International Educational Conference on Assessment 25thJanuary, 2011 Delhi, India
Overview Recent U. S. Developments A System of Quality Assessments Quality Process of Data Use Data Tools Data Use Skills and Culture
Recent Developments • Spotlight on data use in the U.S. for the past decade (e.g., Ikemoto & Marsh & 2007; Hamilton, Stecher & Klein, 2002) • Research suggests that data use is a characteristic of effective schools and districts (e.g., Casserly, 2006; Cawelti & Protheroe, 2001; Consortium for School Networking, 2003; Laird, 2006; Supovitz & Klein, 2003; Teddlie & Reynolds, 2000; Tognieri & Anderson, 2003)
Key Issues • Use of assessment data in the service of teaching and learning requires: • Quality assessments • Quality in the process of using data derived from assessments (Herman & Heritage, 2007)
Three Features of an Assessment System • Comprehensiveness • Coherence • Continuity A comprehensive assessment system “provides a variety of evidence to support educational decision making" (p. 259) A coherent assessment system is built on a well-structured conceptual base – an expected learning progression, which serves as the foundation of all assessments A continuous assessment system provides “indications of student growth over time.” (p. 259). • (NCR, 2001)
Assessments in the System Standards Student Minute -by-minute Daily Weekly Unit Quarterly Annually Adapted from Herman & Heritage, 2007
Assessment Quality • Validity • Reliability • Freedom from Bias
Data Use Process yes Targeted data collection no OK ANALYSIS OF PRESENT STATE Anal Data analysis & interpretation PRESENT STATE TRANSFORMATION PLANNING Identify available data TRANSITION TO NEW STATE Identify what we want to know
Multiple Data Sources • What have my students learned? • What are the strengths and weaknesses student learning?
Multiple Data Sources • Aggregated data from annual assessments • Subscales for more specific information • Quarterly assessments to review rate of progress and patterns
Data Use Process yes Communication Targeted data collection no OK ANALYSIS OF PRESENT STATE Anal Data analysis & interpretation Prioritize needs & options Build consensus & make decisions Identify desired results PRESENT STATE TRANSFORMATION PLANNING Develop targeted plan Identify available data Identify resources Communication Identify evidence-based interventions TRANSITION TO NEW STATE Identify what we want to know Create monitoring & evaluation plan Implement Plans Analyze & rework Collect data for monitoring & evaluation Communication
Going Beyond Assessment Data(Berhardt, 1998) • Achievement • Demographics • Opportunities to Learn • Perceptions
Data Tools • Store • Access • Analyze • Integrate multiple sources • Relational database
Quality School Portfolio (CRESST) 23 different graphic reporting options
Digital Portfolio Store student work with rubric and exemplars
Digital Portfolio Store video samples of student performance
Revised Data Analysis Framework Specific analysis instructions Focus on growth/progress Increased emphasis on student cohorts
Revised Data Analysis Framework Emphasizes patterns/trends Prompts identification of next steps Addresses possibility of needing more data
Skills (Heritage and Chen, 2005) • Ask questions to start the investigation • Frame drill-down questions to deepen the inquiry • Interpretive questions
Skills (Heritage and Chen, 2005) • Select appropriate analysis (e.g., comparative, correlation, cross-sectional, longitudinal analysis) • Interpretive skills • Set priorities and goals
Data Culture • Expectation that judicious use of evidence drives the educational practices of leaders and teachers to improve student learning • Leaders and teachers engage in using data for systematic reflection and planning • “Using data is the way we do things around here.”
To Summarize • Well-aligned system of high-quality assessments • Tools that support analysis and interpretation • Skills to analyze and interpret data effectively • Attitude that data are a valuable resource for guiding improvement