PROJECT RENEW

1. PROJECT RENEW Evaluation Results Summary Sarah Hough

2. Purpose of this presentation • Review the RENEW evaluation as whole. • Focus on effects of project in supporting beginning teachers as they better develop their ability to teach mathematics. • Introduce the summative studies underway. • Talk about what data still needs to be collected in this last year of the project.

3. Evaluation Research What is it? • Basic Research=scientific investigation to develop or enhance theory. • Applied Research= testing theory to assess its ”usefulness” in solving problems. • Evaluation Research=applied research plus a value criteria

4. Usually Evaluation Research is.. A study or series of studies designed for the purpose of assisting project stakeholders in assessing the projects worth.

5. Basics of RENEW Evaluation We are conducting a series of studies that (1) Describe the RENEW model and it’s impact across district contexts; (2) test the effects of the PD activities on participants’ beliefs and understandings of mathematics, pedagogy and equity; (3) determine whether the project had an effect on retention rates in the participating districts.

6. Who gets to decide what is valuable? • Evaluation of RENEW is “client-centered” in that we are interested in the describing the RENEW model from all stakeholder perspectives and examining effects of project activities both seen and unforeseen on all of its participants.

7. Review of RENEW Model Main Goal: Development of a professional development-mentoring model to keep effective teachers of mathematics in the profession Leadership teams offers intellectual and emotional support to Preceptors who in term support the beginning teachers in their schools as they implement National Standards based reform in their classrooms.

8. Objectives RENEW Model National standards-based reform is -- the type of mathematics classrooms envisioned by NCTM) All children have opportunities to work both alone and in collaboration with peers on important mathematics problems., developing both conceptual and procedural knowledge that allows them to be mathematically powerful

9. Work of Preceptors • MEPS --to develop understandings of mathematics, pedagogy and diverse student learners. • Classroom Visitations --to support implementation in classrooms.

10. What Beginning Teachers Need in Order to Implement National Standards-Based Reform • A relevant disposition towards standards-based reform that includes a teacher’s beliefs about teaching, beliefs about learning and beliefs about mathematical knowledge. • The development of deep and flexible conceptual understandings about subject matter. • Pedagogical learner knowledge, relevant to diverse learners, to teach in a way consistent with the standards. Wang and Odell (2002)

11. Does participation in RENEW help preceptees acquire these types of understandings?

12. Beginning Teachers • Particular questions asked and answered by evaluation studies. • Does RENEW have an effect on BT understanding of mathematics concepts, if so, how? • Are BTs beliefs about mathematics and its teaching in line with PSSM (NCTM, 2000)? Does RENEW affect these beliefs over time? • How efficacious do BTs feel in implementing PSSM in their own classrooms?

13. Using Concept Maps to Assess Content Understanding • Amount of knowledge • Way the knowledge is structured

14. Triangulation using reflective writings Every time I attend a math institute I become confident and empowered in math and feel like a better teacher every second I am present in the institute. I never knew statistics could be this fun I’ve already planned most of my 1st month unit in statistics. (Preceptee, Summer Institute)

15. Before RENEW I never felt confident in solving math problems. I rarely understood the WHY behind problems. The project offers a place that I can freely ask and feel comfortable exploring (Preceptee, end of year reflection)

16. Beliefs Scales • Beliefs About Teaching Mathematics (cronbach’s alpha = .76) “In mathematics students cannot understand high level concepts until they have mastered the "basic" steps of a given procedure or algorithm” • Beliefs About Mathematics (cronbach’s alpha = .61)“Doing mathematics mostly involves memorizing facts and procedures” • Beliefs About National Standards cronbach’s alpha = .83“When doing mathematics you are discovering patterns and making generalizations “

17. Beliefs and Understandings

18. Beliefs • After one or more years in the project Preceptee Beliefs About Teaching and Beliefs About Standards-Based Mathematics were not significantly different than that of their Preceptors.

19. Beliefs • Change in Preceptee Beliefs About Teaching and Beliefs About Standards-Based Mathematics were not mediated by number of years in the project. That is teachers in their first 2 years of teaching had beliefs not significantly different than those in their 3rd – 5th years.

20. Triangulation with self-report data asking do you think your beliefs have changed? • 62% stated positively yes, that their beliefs had changed; 16% stated that they had changed somewhat; 18% stated that their beliefs remained consistent but had been strengthened by the project or that they were better able to act upon their beliefs as a result of participation, and 4% stated that no their beliefs had not changed.

21. Absolutely-I have applied many of the ideas and concepts presented in this project. It has radically (I think) changed my mind about the approach to teaching math. (Preceptee, end of year reflection) • I think I always believed in these concepts but Project RENEW has given me the tools to make investigative learning possible in my classroom.(Preceptee, end of year questionnaire)

22. Efficacy and Classroom Practice • Link between efficacy and teaching practice. • Efficacy scale (Cronbach’s alpha = .90) is administered every year.

23. Manage a class of students engaged in hands-on project work Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

24. Help students take responsibility for their own learning Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

25. Lead a class of students using investigative strategies Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

26. Encourage student interest in mathematics Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

27. Use strategies that specifically encourage minorities and females Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

28. Recognize and respond to student diversity Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

29. Changes in efficacy were not mediated by number of years teaching. • Results triangulated with reflective writings

30. I feel much more comfortable about discussing math with my students and realize that there are many ways for them to solve problems. (Preceptee, reflection) • By collaborating with other math teachers, I have learned better teaching practices, as well as strategies to help my students be more successful in math. (Preceptee, reflection)

31. What mediated the change? 1. Effective Preceptors/those with beliefs, knowledge and practice in line with National standards.

32. Relationships What mediated the change? 2. Relationships between people are key to the success of the project.

33. Collaborative Relationships Between Preceptors & Preceptees I have developed a good relationship with my preceptees built on trust and mutual respect. This is something I really didn't think of originally. I also found that our reflective discussions more often than not included issues they were concerned about that were not connected to math, which I felt was okay.

34. How Does Mentor Teachers Leadership Develops Over Time? 3-stage cumulative developmental model Stage I: Preceptor as Resource Stage 2: Preceptor as Relationship Builder Stage 3: Preceptor as Change Agent

35. Stage III: Teachers need to find ways to teach concepts that will help all students be successful. ..I am challenged personally about the equity of our school program. I consistently see the advanced math classes with fewer minority students than our population dictates. I see minority students scoring below proficient on the state standards. I must address these issues personally and share with our MEPS. I must continue to bring up all types of inequity, sometimes gently nudging others to examine their practice. Other times I must not be so gentle!

37. Relationships Between Administrators and Preceptors • Ongoing, year-round Preceptor-Administrator communication • Active Principal participation during recruitment of Preceptees • Visibility of Principals at mathematics professional development sessions.

38. Summative Evaluation in Progress • Tales of Three Districts: Case Studies • Retention Comparison • What are Cohort I Preceptors doing in their settings • Matched Sample “Teacher Burn-Out” Study

39. Technical Reports • Hough, S., Sandoval, N. (2005). An Evaluation of Project RENEW: Year III Report Prepared for the National Science Foundation. February, 2005. • Hough, S., O‘ Rode, N. (2004). An Evaluation of Project RENEW: Year III Report Prepared for the National Science Foundation. February, 2004.

40. Articles • Hough, S; N; Terman, N; Weissglass, J; (2005). Measuring Change in Beginning Teachers’ Statistics Knowledge Networks. To appear: International Journal of Learning, Volume 12. • Hough, S; O’Rode, N; Terman, N; Weissglass, J; (2005). A Respectful Approach to Assessing Change in Teachers Understanding of Algebra: Concept maps. Manuscriptin Revision: Journal of Mathematics Teacher Education.

41. Conference Papers • Hough, S; Sandoval, N; N; Terman, N; Weissglass, J; (2005). Measuring change: Experienced teachers’ develop pedagogical content knowledge by linking content and pedagogy. Paper to be presented at the Annual Meeting of the American Educational Research Association. San Diego, April 2006. • O'Rode,N., Terman, N., Hough, S. (2005). Mathematics and equity: developing the leadership of experienced teachers. Paper presented at the Annual Meeting of the American Educational Research, Montreal, 2005. • Hough, S; Erbes, S; O’Rode, N; Terman, N. (2004). Understanding beginning teachers’ needs: The effects of RENEW, a retention and renewal mathematics project. Paper presented at the Annual Meeting of the American Educational Research, Montreal, 2005.

42. Conference Papers • O'Rode,N., Terman, N., Hough, S. (2004). Mathematics and equity: developing the leadership of experienced teachers to work with beginning teachers. Paper presented at the New Teacher Center Conference, San Jose, CA. • O'Rode, N., Hough, S. (2003). Revisiting the needs of beginning teachers in the context of a mathematics professional development project. Paper presented at The UCSB Education Conference.  Converging Realities: Embracing Diverse Disciplinary Perspectives . Santa Barbara, California, February 2003. • Durias, R., Hough, S., Terman, N. (2002). Leadership for excellence and renewal in mathematics education: An emergent model for supporting beginning teachers in mathematics. Paper presented at the California Commission on Teacher Education.  San Jose, California, March 2002.

43. Next Steps in Evaluation • 2 minute each way dyad: What further information would you find valuable in assessing the worth of the project?