American Educational Research Association Annual Conference Vancouver, British Columbia, Canada

1. Incorporating a Collaborative Assessment Exchange Project with Pre-service Teachers and Administrators: Getting to Know Educator Pedagogy from Two Perspectives American Educational Research Association Annual Conference Vancouver, British Columbia, Canada April 2012 Mary E. Yakimowski and Michael Alfano University of Connecticut – Neag School of Education Neag School of Education’s Office of Assessment

2. Presentation dedicated to mary’s mom – She had always Wanted to be a geography teacher; she ended up as a great “educator” though she didn’t have a teaching certification; she passed away at the age of 93

3. Purpose To determine how we might use an authentic assessment project to positively influence the knowledge, skills, and/or dispositions required of pre-service teacher and administrator candidates (e.g., Wise, Ehrenberg, Leibbrand, 2008) Based on theory of adult learning Collaborative exchange UConn Administrator Preparation Program (UCAPP) Teacher Certification Program for College Graduates (TCPCG)

4. Objectives • Design and implement an exchange project with between two programs across 3 departments • Assess and study the effects of this project on such things as teacher pedagogy and teacher supervision knowledge, skills and dispositions • Interpret the findings and present their implications for both programs

5. Review of Literature Methodology Results Implications of Results Learning's / Future Avenues

6. Review of Literature: A sampling Teacher preparation programs Kagan, 1992; Tang, 2004; Darling-Hammond, Hammerness, Grossman, Rust, & Shulman, 2005; Zeichner, 2005) Early clinical experiences Varrati, Lavine, & Turner, 2009 Integrating administrators Blasé & Blasé, 2004; Brock & Grady, 1998; Vann, 1986; Varrati, Lavine, & Turner, 2009; Cawelti & Protheroe, 2001; Schmoeker, 2001; Sergiovanni, 2001; Varrati, Lavine, & Turner, 2009; Wenger, McDermott & Snyder, 2002

7. Authentic Moore, 2003 Assessment McTighe & O’Connor, 2005; Wiggins & McTighe, 2008). Stiggins, 2009 Adult learning theory Mindful practice includes developing an awareness of one’s mental processing, engaging in active listening, exercising flexibility, identifying bias and judgments, and acting compassionately (Epstein, 1999; McGarvey, 2010).

8. Methodology: Subjects TCPCG – 78 candidates UCAPP – 48 candidates Both affiliated with the Neag School of Education at the University of Connecticut

9. Methodology: Research Design Action research - Mixed methods Overall research question Does this activity enhance candidates’ knowledge, skills, and/or dispositions? Multi-angulation of data; Qual/Quant components Including observations, document analysis, Likert-scale responses,

10. Methodology: Research Design Action research - Mixed methods Overall research question Does this activity enhance candidates’ knowledge, skills, and/or dispositions? Multi-angulation of data; Qual/Quant components Including observations, document analysis, Likert-scale responses,

11. Methodology: Data Sources/Analysis Lesson Plan (TCPCG) Strategic School Profiles, Lesson Plan, Video of Lesson, Reflections, Exchange Form – Part A Document analysis Assessment of Lesson Plan (UCAPP) Exchange Form – Part B e.g., Quantitative analyses of rubric Exchange Project Meeting (TCPCG/UCAPP) Observations Immediate Assessment of Project (TCPCG/UCAPP) Exchange Form – Parts C & D Quantitative descriptive stats; paired t-tests Follow-up Assessment of Project (TCPCG only) Semi-structured focus group

12. Results - Lesson Plans Document Analysis We found that while some candidates listed content as themes or topics, such as “motion’s laws,” others offered a more thorough description, placing the lesson in the context of previous learning and providing a rationale for the timing of the lesson with associated activities.

13. Prompt: “The content that came earlier in the unit was:” “At this point, students have been working on the Personal Narrative unit. We are specifically focusing on Memoirs. Students read a variety of short memoirs to understand exactly what the genre is and is not. They have engaged in the prewriting and writing processes, and they have presented in class with their rough draft. Students were asked to have the entire content of their memoirs in class this day.”

14. Prompt: “Set the stage” for viewing the lesson. Responses often exhibit thoughtfulness. “Force is a difficult concept and as you can see from the many definitions, the students have an idea of what they think force is, however it has a very specific meaning in physics and the student’s initial definitions do not completely line up.”

15. Results – Assessing Lesson Plans

16. Results – The Exchange Meeting Observation First face-to-face meeting Director had each UCAPP student stand to introduce the names of the one or two teachers Parties introduced themselves to each other Dinner line together / Informal discussions Formal meetings convened Culminating activity

17. Teachers • In preparing for a classroom observation by an administrator, ensure that the lesson is “authentic”” • Share what you would like the administrator to focus on during the classroom observation (e.g., As part of the project, differentiation and student assessment was the focus. You could state something about how you plan to incorporate these in your lesson and seek feedback.) • Be open-minded…open to suggestions and willing to accept new ideas • Be self-reflective (e.g., Reanalyze the lesson yourself and share your thoughts on the strengths and areas needing improvement.) • Be willing to seek clarification of what the administrator says in the conference • Do not take “criticism” personally • Have a conversation with the administrator • Assume the administrator has the best interest in mind (e.g., helping you - the teacher - and helping to facilitate learning by the students) • Be respectful of the administrator (e.g., Watch tone of voice; folded arms) • Try to summarize the major points shared by the administrator at the end of the conference

18. Administrators • Have a dialogue before the classroom observation to lay out the expectations for the observation • Have clear expectations of the focus of the lesson (e.g., As part of the project, it was differentiation and student assessment) • Be sure to watch the students, as well as the teacher, during the classroom observation • As an administrator, be confident in your observation and subsequent analyses • Be respectful of the teacher (e.g., Watch tone of voice and be aware of body language throughout the session) • Do not let the ego of being an administrator creep up • Understand that teachers are people who have feelings, too • Be honest • Keep the meeting a mutual conversation – not a monologue • Do not use the “w” word (weakness vs. area for improvement) • Practice “PBIS” • Be specific and help to be a part of the solution (e.g., “What does that look like in the classroom?”) • Use data to back up the areas for improvement and the resulting suggestions • Wrap up the session with clear communication (e.g., Let’s summarize …) • End on a positive note

19. Results – Immediate Assessment

20. Results – Follow-up Assessment • Semi-structured Focus Group • Tell us what you know about UCAPPand initial reaction? • Briefly, tell me how you assessed the students in your class during your lesson? • How did you embed differentiation? • How (in what ways) did the feedback provided by the TCPCG/UCAPP Exchange project improve your teaching? • How did the TCPCG/UCAPP Exchange project prepare you for future interactions with school administrators? • Please share anything else regarding with Exchange project that we have not addressed.

21. Results – Follow-up Assessment • How (in what ways) did the feedback provided by the TCPCG/UCAPP Exchange project improve your teaching? • Positive responses: Levels of comfort, detailed feedback, provision of insight/resources for future use, impartial reviewer, more tailored feedback • Negative responses: Issues with the timeline, lack of feedback

22. Overall – Findings Give Hope Participants from both programs generally felt it was a positive, enjoyable experience with clear and evidence-based feedback/recommendations offered The last activity of the dinner meeting was found particularly helpful to many of the participants as items to remember in the future when facing a teacher-principal evaluative interaction

23. Parameters All data collection needed to be done within the framework of designated courses and timeframe Participants were from one institution Did not include either all TCPCG students, or the other teacher education program at that institution Researchers needed to respect the regular course load of these students, particularly in TCPCG as participants are in an intensive one-year program

24. Learnings/Future Avenues • Clearer explanations of the project • More immediacy of feedback • Expand to the other TCPCG cohort and perhaps to the other teacher education component • Examine the standardized assessment results that UCAPP students take • Schedule alumni survey, embed some questions on the long-time potential benefits

25. Conclusions • Much time for all stakeholders - Take it slowly and cautiously • As higher education faculty, adult learners are mindful practitioners • Learning process did provide authentic learning opportunities (i.e., delivering a lesson, evaluation and giving feedback to a lesson)to support students’ self-esteem, self–efficacy, self-directedness • Self-directed learning allow students to be involved in what and how they learned(e.g., reflecting on the experience and dialoguing with others promoted collegiality across participants from the two programs in a positive way) • Scholarly significance - A collaborative assessment could foster in future teachers and administrators that learning is a continuous process that will further evolve and develop