Literacy Clinical Teacher Preparation that is Transformative

1. Literacy Clinical Teacher Preparation that is Transformative Literacy Research Association Dallas, TX December 4, 2013

2. Four Distinct Research Studies • Transfer & Transformation of Teachers in Clinic: Longitudinal review of the 30 Cases across 6 years. Collective Case Study Approach • Video for Assessment: Use of video clips to analyze students’ reading performance by experts, graduate students and novices. Formative Design Study of Assessment Protocols & Rubrics • Video for Teacher Reflection: Analysis of video tasks across sites Design-based in Year #1; This year: Cross-Case Analysis • iPad Use in Clinics Mixed methods (qualitative & quantitative) across five sites.

3. SUMMARIZING ACROSS SITES • Teachers take what they learn in Reading Clinic into their classrooms/schools. • Analyzing videos of children’s reading, with guidance from rubrics, is helpful in assessing teachers’ knowledge of literacy processes. • Using video with specific directions, debriefing, and collaborative inquiry deepens teachers’ reflections. • Technology use by teachers is becoming more pedagogically powerful in Reading Clinics.

4. Literacy Clinic cases Evan Ortlieb, Monash University (Australia) Julie Gray, University of Virginia (Virginia) Tammy Milby, University of Richmond (Virginia) Barbara Laster, Towson University (Maryland) Stephen Sargeant, Northeastern State University (Oklahoma) Literacy Research Association Dallas, TX December 4, 2013

5. History of the cases Project • 2006-2007 • Interview study of 28 graduates • Identified 5 areas of clinic transfer (instruction, assessment, coaching, leadership, technology) • 2009-2010 • In-depth interviews of nine clinic graduates • 2010-2011 • Add nine additional graduates, including two new sites • New projects based on transfer/transformation findings • 2011-2012 • Add five additional graduates, continuation of work on “Transfer” and “Transformation” • Research Team Examines: • Habits of Mind • Clinical Experiences that Matter & Recommendations • Disjunctures • 2012-2013 • Three additional cases added, review of all 30 cases in entirety

6. Purpose/Rationale • Literacy Clinic: Research to practice- • Explore ways in which clinic/lab graduates transfer clinic/lab practices to schools • Explore ways in which graduates take on literacy leadership roles in schools • Understand how the clinic/literacy lab experience supports literacy leadership • Investigate current instructional & assessment practices transferred including national trends • Understand clinic/lab role in multiple paths to leadership

7. Context & theory • Roles of literacy professionals changing (Bean, et al., 2002) • Coaching and leadership in the forefront (Walpole & McKenna, 2004) • Little research on preparation of literacy professionals (Anders, et al., 2000) • Growing condemnation of teacher preparation (Darling-Hammond, 2000, 2006; Duncan, 2009) • Leadership is a key component of educational reform (Middlebrooks, 2004) • Teachers need support to navigate mandates, enhance skills as literacy leaders/coaches, & reflect on best practices (Ortlieb & Cheeks, 2013) • Research needed on literacy professional preparation that leads to both effective teaching and effective leadership • Training vs. Teaching (Hoffman & Pearson, 2000) • Guided practice opportunities are essential

8. RESEARCH QUESTIONS • 1.) What transfers from the literacy lab to educational contexts? • 2.) How have trends, leadership, and disjunctures changed across time?

9. Methodology • Identical methodology across new & existing sites • Maryland (1 site) • Virginia (2 sites) Each researcher followed a 3 phase process: • Phase 1 • Initial screening interview • Collection of artifact to represent practice (graduate chosen) • Screening observation • Phase 2 • In-depth interview • Targeted observation • Phase 3 • Follow-up/Retrospective interview

10. DATA ANALYSIS • 3 Phases using Constant Comparative Method (Glaser & Strauss, 1967; Strauss & Corbin, 1990) • Phase 1: Identified instances where graduates focused on • Aspects of clinic/lab that supported their development (Clinic Experience) • Current practices in classrooms that drew from participation in clinic (Transfer, Transformation) • Phase 2: Grouped instances from Phase 1 into like categories within clinic experience and transfer; created codes for categories • Phase 3: Confirm/reconfirm categories (Miles & Huberman, 1994); collapsed into broader themes • Member checking

11. 2012-2013 Findings : Text Selection/Text Types Program Fidelity versus Student Needs Beginning of Common Core Implementation Assessments Instructional Insights Confidence Flexibility & Masterful Reflection

12. Participant quotes: Amy, Reading Specialist, Maryland • They took Bud, Not Buddy—which is a6th grade text—and bumped it down to 4th grade. It seems as though they are just taking what was a higher grade curriculum and putting it down to a lower grade—and then they call that “rigor.” • My idea is to have our Guided Reading as the heart of our reading program here. Then, we will have accessible texts for all students on their instructional level. Because these CCSS texts are very frustrating for these students, so how are they progressing if they are frustrated?

13. Participant quotes: Paula, Classroom Teacher, Grade 2, Virginia • As a beginning teacher last year, I felt totally overwhelmed. School let out at 2:00pm, but I never left before 6:00pm. Our school was on the “warned list” due to our performance and someone from outside [the district representative] was always coming in. I survived because of collaboration. There was this revolving door of new initiatives all the time. After the tutoring class, I knew I needed support from others who ‘understood’ and I reached out to the Title 1 Teacher & Reading Specialist. • For our curriculum, we implemented a balanced approach to literacy. We do whole group, small groups (guided reading & words study), word study and writing. I felt comfy with this format due to the approach we learned in clinic.

14. Participant quotes: Miranda, Reading Specialist, Virginia • I decided to leave the classroom and move into a leadership role because of the mentoring and confidence I gained [from clinic]. We learned every type of assessment and intervention… Tutoring students helped me look at reading with diverse learners [Lengthy discussion naming different assessments learned]. It helps me to know how to do more rich assessments with students now. • I was always very interested in ESL students… our class allowed me to work one-on-one tutoring a student whose home language is Burmese and I also worked with a student with a disability. Both clinic experiences prepared me for how to work with small groups and prepared me for the variety of needs and challenges I face now as a Reading Specialist in this urban setting. After the clinic, I decided to go ahead and complete my ESL endorsement.

15. Longitudinal review of the 30 Cases across 6 years, Collective Case Study Approach • Approach: • Seek IRB approval • Define design (Merriam (1988); Miles & Huberman (1994) & Barone (2004): • Particularistic: • Cases are all focused on ‘Literacy Clinic’ transfer categories determined through previously research* (instruction, assessment, coaching, leadership, technology). • Descriptive: triangulation provides rich description of common trends • Heuristic: continued study of existing data will enrich understanding of what transfers from clinic across time • Inductive: Data drives the understandings which are emerging • Examine existing “cases” data & findings to complete an in-depth analysis of trends (categories, coding). Seek cases containing information-rich informants. • Check for accuracy & misconceptions by consulting with key informants and/or researchers. • *Derived from interviews of 28 additional clinic graduates

16. SUMMARIZING ACROSS SITES Does clinic make a difference compared to other coursework approaches? Collective review of clinic graduates report that the literacy clinic helped them in similar ways: • Student-centered, differentiated instruction • More of a focus on strengths & needs • Variety of “assessment practices” incorporated in the classroom • Collaboration & Sharing with others • Coaching/leadership opportunities • Working through disjuncture/policy changes/mandates successfully or deciding to change paths • Advocacy & implementation for research-based practices • Developing deep and thoughtful beliefs about literacy • More “masterful” teaching approach, habit of mind • (reflective practice, asking why?, seeking excellence

17. In Conclusion: “[Clinic]really puts it all together when you focus on one student—use data and plan intervention that matches—a real eye opener. Now when I analyze data I can see the big picture…Writing the practicum report actually…showed how to triangulate the data, helped me write a more cohesive report, helped me look at individuals as well as the classroom. It helped me recognize trends, organize—prioritize instruction and made me more confident so when I write a report or meet with a teacher, I really know what I am talking about and can now explain it in “real words” and get down to what learning needs to take place.”

18. Video Protocols for the assessment of Teacher Knowledge and Skill in Literacy Assessment and Instruction Literacy Research Association Dallas, TX December 4, 2013

19. Team Members

20. Purpose/Rationale • Develop an authentic assessment to measure reading teacher/specialist candidates’ abilities in assessment and planning instruction • Provide a tool aligned to instructional pedagogy • Enhance multiple-choice format state certification exams

21. Video for Assessing Teacher Knowledge and Skills: RESEARCH QUESTIONS • How can literacy faculty develop a protocol that can be used across the nation, when multiple forms and analyses of assessment are used? • How can we authentically assess teachers/specialist candidates’ ability to analyze student reading? • How can we develop a reliable and valid rubric as a tool to evaluate teacher knowledge and skill in assessing videos of a student’s reading?

22. Experimental Design:Formative Research • Intended to improve instructional theories, models, practices, and processes (Bradley & Reinking, 2011; Brown, A. 1992). • Focuses on the characteristics of Reinking and Bradley’s (2007) formative design: • Established educational goals based in theory • Implement an intervention to achieve goals (protocol) • Collect data to identify factors enhancing or inhibiting achieving goal • Modify intervention based on unanticipated factors • Note how intervention changed • Determine positive and negative unanticipated effects of the intervention (to be determined)

23. Methodology: Video Development • Video protocol Instructions • Authentic text • Introduce the passage • Student reads the passage • Request oral retelling • Probe hesitant reteller • No published assessment text

24. Hungry Animals Text Reading Recovery Level 11 Middle First Grade

25. Graduate Student Video Response

26. Declaration of Independence: Amazing Days of Abby Hayes 4.3 GE Lexile 510 Guided Reading Level Q.

27. Methodology Video Response Protocol

28. Methodology Mixed Methods Qualitative Methods • Rubric: Researcher Developed • Researcher “answer keys” completed with comparison of responses of all six independent responses • Benchmarking: All six researchers used rubric on one set of teacher papers • Rubric adjusted to reflect experience in application to the teacher papers • Rubric “tested” by partners on one set of papers • Rubric again revised • Revision: Teams of two researchers used rubric to score each teacher’s paper independently • Discrepancies resolved

29. Methodology-Continued Mixed Methods • Quantitative Methods • Descriptive data presented by class sets and combined data set • Correlations of subscores and totals in combined data set • Data consolidation

30. Participants (Contexts of Sites) • Southeastern United States (1Case) • Administered at beginning of the semester • Undergraduate students in senior year • In ⅘ sequence of reading endorsed courses • Central United States (2 Cases) • Administered at the beginning of the semester • Graduate students at the beginning of the program- some with no knowledge of IRI’s and Running Records • Graduate students during first clinical course • Western United States (4 Cases) • Administered at the beginning and end of the term (pre- and post-assessments) • Undergraduate students in a Literacy Lab course • Had completed two other literacy courses including IRIs and running records

31. Current Protocol Rubric

32. Current Protocol Rubric Continued

33. Data ConsolidationHistogram N=106

34. Data Analysis-Case 1Southeastern United States

35. Data Analysis-Case 2CENTRALHA EARLY Program

36. Data Analysis-Case 3Central HA BEGINNING CLINICAL

37. Data Analysis-Case 4 Western AH Pre

38. Data Analysis-Case 5 Western AH Post

39. Data Analysis-Case 6Western HA Pre

40. Data Analysis-Case 7 Western AH Post

41. Conclusions • Creating a reliable and useful rubric requires a commitment of significant time • Inter-rater reliability • Trial applications to candidate responses • Multiple revisions • Individual rubrics must be created for each video sample • Current data resulted in “normal distributions” of scores • Given this sort of development it may be possible to take this approach to assessment “to scale” • Latent semantic analysis could be used for initial machine scoring • Readings of individual protocols may be necessary

42. Next Steps • Performance task of this nature is very promising • Capturing good quality videos requires careful planning and recording • Collect 4 new videos with revised protocol (primary, intermediate, middle and high school) • Develop specific rubrics for each • Administer it as a pre-test and post test for undergraduate and graduate students

43. References Bradley, B., & Reinking, D. (2011). Revisiting the connection between research and practice using formative and design experiments. In N. Duke & M. Mallette (Eds.), Literacy research methodologies handbook (2nd ed.). (pp. 188-212). New York, NY: The Guildford Press. Brown, A. L. (1992). Design Experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141--178. Creswell, J. W. & Plano Clark, V. (2011). Designing and conducting mixed methods research, 3rd Ed. Thousand Oaks, CA: Sage. Greene, J., Caracelli, V., & Graham, W. (1989). Toward a conceptual framework for mixed-methods evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. Johnson, R. B. & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26. Onwuegbuzie, A. & Mallette, M. (2011). Mixed research in literacy research. In N. Duke & M. Mallette, (Eds.). Literacy research methodologies handbook, (2nd ed.). New York, NY: The Guildford Press. Reinking, D., & Bradley, B. A. (2008). On formative and design experiments: Approaches to language and literacy research. New York: Teachers College Press.

44. Videotaped Analysis for Teacher Reflection Literacy Research Association Dallas, TX December 4, 2013

45. Participants • Terry Deeney, University of Rhode Island • Cheryl Dozier, University at Albany • ZubeyirCoban, University at Albany • Barb Laster, Towson University • Jeanne Cobb, Coastal Carolina University • Marcie Ellerbe, Coastal Carolina University • Debbie Gurvitz, National Louis University • Anne McGill-Franzen, University of Tennessee • Natalia Ward, University of Tennessee • Jennifer Lubke, University of Tennessee • Mary McVee, University at Buffalo • Ashlee Ebert Campbell, University at Buffalo • Liz Tynen, University at Buffalo • Erica Bowers, University of California at Fullerton

46. Theoretical Framing • Dewey (1933) reflection is “active, persistent, and careful consideration of belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it ends” (p. 9) • Schon (1983) analyzing and acting purposefully on a situation with the goal of changing it; developmental process • Reflection on action • Reflection in action • Reciprocal reflection in action • Renewed interest in using video as a reflective tool (Grossman, 2005)

47. Video reflection as a tool for improving teaching • Research suggests video reflection can be an effective strategy to help teachers improve their teaching (Penny & Coe, 2004; Tripp & Rich, 2012) • In our transfer/transformation study (Deeney et al, 2011) clinic participants named video reflection as a powerful tool for improving their practice • Studies vary on ways in which reflection was structured [Tripp & Rich, 2012(a); 2012 (b)]

48. Dimensions of Video Analysis (Tripp & Rich, 2012, p. 681)

49. Purpose/Rationale • Systematic look at how videotaped reflection tasks are structured in clinics/labs across the country • Analysis of collegial and instructor feedback across the country • A beginning analysis of how clinic participants respond to video reflection tasks

50. Research Questions • In what ways do clinics across the country have teachers use video to reflect on their own teaching? • What practices do clinics/labs across the country use to facilitate reflection on videos of one’s own teaching? • How has video-facilitated reflection affected teachers’ understanding of their own teaching?