1 / 12

The Role of a Theory of Action for Educator Evaluation Systems

The Role of a Theory of Action for Educator Evaluation Systems. Erika Hall Center for Assessment Presentation at the 2014 National Conference on Student Assessment New Orleans June 25, 2014. Common Design for Educator Evaluation Systems. Figure from: Diaz-Bilello & Hall, 2014.

bebe
Download Presentation

The Role of a Theory of Action for Educator Evaluation Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Role of a Theory of Action for Educator Evaluation Systems Erika Hall Center for Assessment Presentation at the 2014 National Conference on Student Assessment New Orleans June 25, 2014

  2. Common Design for Educator Evaluation Systems Figure from: Diaz-Bilello & Hall, 2014 Hall, Theory of Action for EE Systems, NCSA. 2014

  3. Ways in Which Systems Might Vary Hall, Theory of Action for EE Systems, NCSA. 2014 • How measures are operationalized and calculated • The manner and precision which the results associated with a measure are represented to support reporting and aggregation • The process used to combine measures for the purpose of establishing a component or final effectiveness rating • The weights given to different types of measures when defining a component or final effectiveness rating; • Rules applied to teachers associated with tested vs. non-tested grades and courses; and • The flexibility afforded to districts in terms of how different components of the system are selected and implemented

  4. What is a Theory of Action Hall, Theory of Action for EE Systems, NCSA. 2014 An empirically and logically stated argument A set of underlying assumptions A testable hypothesis …that outlines how and why a given system, as designed, will support the achievement of specified goals.

  5. Pieces of the TOA Puzzle • Goals/Expected Outcomes • Intended Uses of System Results • Theory as to manner in which the system will bring about desired change (System-Level TOA) • Components of the system, including: • the rationale for their selection • the goal(s) they are intended to support • How they are intended to relate to one another • Mechanism by which components are intended to provide for specified goals. (Component Level TOA) • Inferences/assumptions underlying the system working as intended. (Interpretive Argument). Hall, Theory of Action for EE Systems, NCSA. 2014

  6. Why is it Important to Develop a TOA Supports the development of a common language & shared understanding. Clarifies the rationale for the system design Helps to identify areas of inconsistency/lack of coherence Provides for a document that can be used to “defend” the thoughtfulness and coherence of the system. Supports the specification of a plan for validation that is appropriate for your system. Hall, Theory of Action for EE Systems, NCSA. 2014

  7. Figure from Hall, 2014

  8. Simplified Example Hall, Theory of Action for EE Systems, NCSA. 2014 • Purpose: Administrative – Support administrators in making retention, promotion, or dismissal decisions • Goal: Remove all non-effective teachers from the classroom • Use of Results : Identify high and low performing teachers • Claims to be supported: • Score-Based: Teachers who receive an Overall Rating of “Not Effective” do not display the skills and practices expected • Impact-Based: System will provide administrators with the data to inform decision making

  9. Simplified Example Hall, Theory of Action for EE Systems, NCSA. 2014 • Inferences/Assumptions • Overall Effectiveness Rating is an accurate, reliable measure of expected skills and practices • Administrators are provided with system data in time to support decision making • Evidence • Feedback from evaluators regarding the perceived appropriateness of the procedure used to define and establish the overall effectiveness rating • Feedback from administrators regarding the availability of data to support decision making

  10. Additional Benefit of TOA Hall, Theory of Action for EE Systems, NCSA. 2014 • States differ with respect to manner and degree to which there is local control over EE system design. • A clearly defined TOA supports states in specifying rules related to the development and evaluation of alternate evaluation systems, or system components. • Supports states in identifying what is/is not eligible for modification • Articulates, by example, the type of documentation necessary to support the development/use of an alternate system.

  11. Flexibility Continuum Relative to TOA Hall, Theory of Action for EE Systems, NCSA. 2014

  12. Thank You!!! Hall, Theory of Action for EE Systems, NCSA. 2014 References: Hall, E. (2014). A Framework to Support the Validation of Educator Evaluation Systems. A paper presented at the National Council on Measurement in Education. Philadelphia, PA. Diaz-Bilello, E. & Hall, E. (2014). Considerations for Establishing Performance Standards for Educator Evaluation Systems. A paper presented at the National Council on Measurement in Education. Philadelphia, PA. Erika Hall ehall@nciea.org

More Related