RENEW Evaluation

1. RENEW Evaluation Sarah Hough, Ph. D. & Niki Sandoval, M. A. University of California, Santa Barbara

2. Purpose of RENEW Evaluation • Are implementation goals met? • Are project activities moving participants toward the specific pedagogical objectives of • Increasing participants’ understanding of mathematics • Increasing participants’ ability to implement national standards in mathematics • Increasing participants’ understanding of equity issues and their ability to work with English Learners • Does the project help to retain teachers in the profession?

3. RENEW Grows

4. Current Participants • 136 preceptees • 110 new teachers attended summer institutes • 36 preceptors • 57 schools • 9 districts

5. Activity Increases

6. What New Teachers Say Every time I attend a math institute I become more confident and empowered in math. I feel like a better teacher every second I am present in the institute. summer institute participant I feel more confident when analyzing statistical data now based on the activities we did as a group. summer institute participant Cross institute feedback in terms of • growth in leadership • What beginning teachers said they gained from the institutes. • How many of the BTs signed up for participation in RENEW • How many returning preceptees from previous years

7. Juggling Pedagogical Content Knowledge Efficacy Beliefs About Teaching Instructional Strategies PEDAGOGY Curriculum Student Diversity Beliefs About Math Differences in Student Thinking SubjectMatter STUDENT CONTENT

8. Collecting Data About Beliefs • Each year renew beginning teachers fill out a questionnaire that includes: • Two graded response scales that measure beliefs about mathematics and beliefs about pedagogy

9. Beliefs About Mathematics and it’s Teaching Before Participation Math as investigation Not in line with PSSM In line with PSSM Math as procedures Teaching as “telling”

10. Change in Preceptee Beliefs Math as investigation ß Math as procedures Teaching as “telling”

11. Collecting Data About Learning Content • Pre and post “statistics” concept maps from 22 beginning teacher participants at the UCSB institute were analyzed for change. • Content Analysis--themes of institute were: measures of central tendency, graphical representations, probability, sampling and bias.

12. Content Analysis of Maps Statistics Representing concept Venn Diagrams Graphs Tables T-charts Pie-charts Bar charts Chunk

13. Math Content Analysis of Maps

14. Example 3-pre map Statistics Mode Most Mean Median Bar graphs Frequency Average Middle Histogram Baseball players When do we really use this? Races

15. Example 3-post map Analyze Data T-charts Statistics Graphs Venn Diagrams Bar charts Big 3 Pie-charts Mode Most frequent Median Mean Bar graphs Stem and leaf Average Middle Reinforces results Not always useful Box and whisker

16. Changes in Subject Matter Structure

17. Participant view of their own learning • When participants were asked to reflect on their learning using the pre and post concept maps the majority of them agreed that they had added more concepts to their maps “I ran out of room for all of my thoughts on my 2nd concept map! What a great visual assessment of my learning. I hadn’t realized just how much I really did learn this week until I saw the difference between Monday’s and Friday’s maps. ”

18. Participant view of their own learning • And, that their maps were more sophisticated and connected “My first graph does not have as many connections with the sub categories as the second one. I have a deeper understanding of topics now, for instance, that the mean could get affected by a very big number , while the median could stay the same.”

19. Change in PCK--Vertical Curriculum Knowledge “Prior to this week’s activities, I attributed statistics more to a college level or advanced math level. However, now I understand and am excited to see how statistics applies to the elementary level of teaching and learning.”

20. Change in PCK--Horizontal Curriculum Knowledge “What I have come away with is not only a better understanding of the content of statistics but more importantly, how to integrate it into my classroom. I can honestly say statistics was not a topic I covered in my first year of teaching but I feel I now have many ideas/topics that would be relevant to my grade level as well as have student interest. am very excited and plan to use Math By All Means –Probability Grades 1-2. this year I plan to use the textbook less and use Math By All Means to supplement the lesson ideas. ””

21. How RENEW Fosters PCK • Math Activities • Classroom connections (e.g. Good Questions for Teaching Mathematics) • Understanding English Learners (e.g. Language oppression and implications for learning of math)

22. Change in PCK--Understanding of Student Diversity “Bias is a concept on my second map, but not on my first. I had heard bout statistics being “used” to support an opinion but did not truly understand how the same information could say such different things. In the activity looking at the work force comparing gender and race, though are groups were given the same data, the results were quite different. Looking at sampling size, locations, etc. makes a huge impact on what the statistics on a given topic may say. “

23. Change in PCK--Understanding of Student Diversity “One thing that was illustrated a lot was how differently people think about the same thing, how we each approach and solve problems in various ways. Its great to learn different perspectives by working with others. The language and statistics activity was great for emphasizing how much more I need to be conscious of possible language barriers in my classroom as opposed to the students not understanding the material. I’m really glad that you incorporate social issues like race/ethnicity and sexism.”

24. Collecting Data About Efficacy Each year renew beginning teachers fill out a questionnaire that includes: • Six graded response questions about efficacy

25. Efficacy to implement effective teaching practices • Manage a class of students engaged in hands-on project work Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

26. Efficacy to implement effective teaching practices • Help students take responsibility for their own learning Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

27. Efficacy to implement effective teaching practices • Recognize and respond to student diversity Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

28. Efficacy to implement effective teaching practices • Use strategies that specifically encourage minorities and females Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

29. Efficacy to implement effective teaching practices • Lead a class of students using investigative strategies Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

30. Efficacy to implement effective teaching practices • Encourage student interest in mathematics Somewhat prepared Very well prepared Not adequately prepared Fairly Well prepared

31. Retention • 93% of RENEW beginning teachers are still participating in RENEW after 1 or more years in the project.

32. How is RENEW developing PCK in BTs • By providing intellectual and emotional support to the Preceptor cadres with whom they work • By providing professional development that focuses on leadership to these experienced teachers. • The project is collecting data using multiple methods to trace the development of leadership capacity in preceptors

33. RENEW Evaluation on the web http://renew.education.ucsb.edu Research and Evaluation Papers and Presentations, Reports