Independent (outside) Review of Performance Assessment Instruments and Scoring Rubrics

1. Technical Assistance Training Independent (outside) Review of Performance Assessment Instruments and Scoring Rubrics

2. Caveats • A level of detail that I can’t get to unless you have specific questions. • My experience comes to you as a professor developing educational administration curriculum and as an ELCC (ISLCC) accreditation reviewer of ed admin programs. • Performance is the new standard. • An applied reality to get approval--reviewers will hone in on performance quality. 5. All programs in the state utilize MDE standards--Principal and CO/Supt

3. Exemplars • Describe what exemplars will be used to show each possible score point. (For example, if the score point is “pass/fail,” what exemplar would demonstrate “barely passing” and what exemplar would demonstrate “barely failing”?)

4. Training Raters • Describe how the association will train raters to use exemplars and rubrics in the assessment process and will monitor scoring and recalibrate raters so they do not deviate from scoring guidelines

5. Assessment System • Establish a record-keeping system to maintain auditable assessment decisions.

6. Independent Review of Performance Assessment Evidence • Educational leaders at all levels would be eligible for such a panel, with the exception that they should: not be faculty in the program, not be responsible for delivering content and not be officers or staff of the association. • These could be contractors or volunteers and would have some planned term of service, eg. three years, without substitution. • Ideally, at least two of the several members would have some systematic experience in designing or evaluating performance assessment (not with MEAP tests). These could assist the assessment coordinator in developing the framework for assessment tasks and for collecting necessary evidence of reliability and validity.

7. Performance Assessments Are: Essential, not arbitrary or contrived to shake out a grade. Enabling, constructed to point the candidate toward more sophisticated use of skills or knowledge. Contextualized complex intellectual challenges, not atomized tasks corresponding to isolated actions. Representative challenges designed to emphasize depth more than breadth. Engaging and educational. Involved with broad tasks or problems that may be somewhat ambiguous. (NPBEA Standards, 2002, p. 12)

8. An Example of Performance Experience Required of a Student • Organizational Management & School & Community Relations • ELCC 3.1, 3.2, 3.3, 4.1, 4.2  • I. Collaborate with a district leader to assess (evaluate) the district’s facilities including central office buildings, individuals school facilities, central office budgets, individual school budgets, school schedules, and school staff members.  • II. Determine how adequately the school facilities addresses/promotes learning needs of the classrooms (3.1, 3.2, 3.3,) • 1. Conduct a needs assessment of classrooms and equipment, • 2. Conduct a needs assessment of facility in relationship to learning • 3. Determine how adequately the facilities promote technology in the classrooms (3.1, 3.2, 3.3, • 4. Assess the functional technology in the classroom (infrastructure, • 5. Assess the use of the technology. • 6. Determine how adequately the central office budget and individual school budgets address school improvement goals and the vision of the schools, (3.3) and • 7. Develop a strategic plan for budgeting linked to school improvement

9. Performance Activity: Handout Distributed Leadership

10. ORGANIZATIONAL MANAGEMENT RUBRIC & SCHOOL AND COMMUNITY RELATIONS

11. From NCATE: Guidelines for Reviewers A. Do the assessments assess meaningful cognitive demands and skill requirements at challenging levels for candidates? • COMPLEXITY—Are the assessments congruent with the complexity, cognitive demands, and skill requirements described in the standards? • DIFFICULTY—Is the level of effort required, or the difficulty or degree of challenge of the assessments, consistent with standards? Is this level reasonable for candidates who are ready to teach or to take on other professional educator responsibilities?

12. Is it Activity or Performance? • SPECIFICITY—Are the assessments vague or poorly defined? The assessments might include an entry like “portfolio entries, test results, observations.” What entries? What test results” What observations? These need to be identified as specific experiences. Is the assessment information oblique or confused? Sometimes the response does not actually address the standard.

13. Closure • Under the guidelines established for educational administration programs that are seeking Educational Leadership Constituent Council (ELCC) accreditation through the National Council for the Accreditation of Teacher Education (NCATE), performance means that a student must accomplish some kind of meaningful task that teaches or enhances a relevant or realistic skill or ability that a school administrator might perform in the day-to-day operation of a school.