What We Know “So Far” About Aligning Teacher Preparation with P-12 Student Learning

1. What We Know “So Far” About Aligning Teacher Preparation with P-12 Student Learning Funded by a STEP grant from AACTE Indiana University School of Education Indianapolis

2. Project: Standards-based Assessment Project - (2006) • PI: Linda Houser. • Project Director: Linda Houser. Co-investigators: Signe Kastberg, Jenny Conner, and Paula Magee • Funder: American Association of Colleges of Teacher Education Indianapolis Public Schools ($15000.00). Project timeline: September 2005-October 2006

3. Abstract:This grant provided support for the School of Education to continue its assessment work in the areas of elementary mathematics and secondary English. Assessment data from Benchmark II, which addresses mathematical knowledge as it relates to place value, and P-12 student learning will be analyzed to determine if the benchmark assessment is a good predictor of our graduates’ teaching skills and ability to impact their own students’ learning. In this study, graduates of the elementary teacher education program at IUPUI, for whom there are Benchmark II data, were invited to participate in seminars addressing the teaching of mathematics and to design a mathematic teaching unit and accompanying assessments. The graduates collected student work completed prior to and after the unit which allowed their students to demonstrate conceptual understanding of place value. This student work will be analyzed to determine the changes in the children’s conceptual understanding of place value. Data from classroom observations will be analyzed to determine the graduates’ conceptual understanding of place value and to determine their pedagogical content knowledge. Connections among Benchmark II data, data collected about the graduates’ teaching, and data from the P-12 student work will be investigated.

4. The second area of assessment this grant supported was the development of an assessment activity for our secondary English students to assess whether they can analyze teaching as it relates to their content areas. The faculty believe that it is important for candidates to be able to analyze good and bad teaching techniques as demonstrated by others and to reflect on the impact of the teaching on student learning. This is the first step to analyzing their own teaching and its impact on their students. The grant provided support for piloting an activity which had the education major watch a video of a lesson in their content area and answering questions relating to the lesson.

5. Using Benchmarks for Assessment • Benchmarks -Assessments to determine if candidates can take the skills, knowledge, and dispositions they have learned from their courses and “put them together” and “teach.” • Performance-based • External to class assessments • Occur at key points as candidates move through the program.

6. Elementary Benchmark II • At the end of Block II of a four-block program • 90-120 per semester • Used by the unit for 6 years • Grounded in conceptual understanding of place value • Rubric developed and used to answer three focus questions • Has provided evidence for programmatic change early in its use • Submitted electronically

7. Benchmark II • The performance task is designed to assess interns’ abilities to analyze children’s conceptual knowledge and to demonstrate that the interns can engage learners in a two-way conversation that allows the interns to assess the children’s grasp of a mathematical concept. Interns are asked to identify good follow-up experiences for the learner and self- assess their own effectiveness as an interviewer.

8. Guiding Questions • The intern’s mathematical knowledge has the potential to support student thinking about mathematics with understanding. • The intern is beginning to understand how to assess student thinking using interviews. (attends to student responses, bases comments on evidence from data, uses questions to probe student thinking). • The intern has intellectually engaged in making sense of material from Block I & II (respect for students, child centered, bases follow-up on evidence).

9. Moving Forward A Step Grant • Investigation of the degree to which Benchmark II is a predictor of implementation of skills, knowledge and disposition when teaching. • Determining candidates’ impact on student learning at the level of conceptual understanding.

10. Participants • Eight graduates of the elementary teacher education program • Teachers in grades K-5 • Benchmark II data available • Some did not “pass” the Benchmark II • Follow-up data available

11. Data Collection • Data from Benchmark II • Data from Benchmark II follow-up • Standards • Assessments Tools • Lesson Plans • Student Work • Observations

12. JulieExcerpt from Benchmark II Next, I wrote 467 on a piece of paper and asked her to use the manipulatives to give me that particular amount. Using different representations indicates her knowledge of place value, again because of grouping the same amount in different ways.

13. Julie-Standards • 2.1.2 Identify the pattern of numbers in each group of ten, from tens through nineties. • Example: What pattern do you see on a hundreds chart for the numbers 12, 22, 32, etc.? • 2.1.3 Identify numbers up to 100 in various combinations of tens and ones. • Example: 32 = 3 tens + 2 ones = 2 tens + 12 ones, etc. • 2.1.5 Compare whole numbers up to 100 and arrange them in numerical order. • Example: Put the numbers in order of size: 95, 28, 42, 31.

14. Julie-Assessment Tool

15. Pre-Assessment Data

16. Pre-Assessment Data

17. Activity This activity was taken from the Indiana Mathematics Grade 2 Standards Resource

18. Observation Data “We are doing dime, penny, dime, penny.” (child has positioned the manipulative in a pattern dime first followed by a penny). “Do you think you will get there if you do dime, penny, dime, penny?”.....”Did you do it (make 35 cents) using your pattern?” child counts ten, eleven, twenty, twenty-one, thirty, thirty-one, thirty-two, (3 dimes and 7 pennies) “If you group the dimes together will it still be 35?” (Julie walks away and M (child’s name) approaches illustrating that if you begin with 39 cents and “take away” 4 cents you have 35 cents.) Julie says “M(child’s name) you are amazing.” “ (to another group) What did you guys find out? How many different ways did you find?” “One dime and 25 pennies.” “What did you get?” (reads another student’s work) “I put all of the pennies together and that made 35.”

19. Post-Assessment Show me 34 using any of those manipulatives. What did you do? C: I actually got 3 tens and 4 cubes. Is there any other way you can show 34? C: Yes How C: I take the 3 tens and I still have the 4 cubes. 5, 6, 7 Wait, before you count them out, how many cubes are you going to need? (no response) If you were going to trade these in, how many would you need? (she doesn’t know) Count them and let’s see if we can find out. (counts by 1’s to 34 from 4) (counts the cubes 1-30) C: 30. Is there any other way you can show me that number? C: No

20. Reflection on Student Learning When asked to show me more than one way using the base 10 blocks, she showed me the ‘normal’ way and using only ones. She did not think to trade in some tens for ones. She has to touch count all the cubes to 34. An interesting thing she said when asked about 2 tens and 14 ones is, well, first she said it looked like 214 on the paper, then she said that it was 16. She added 2 to 14.

21. Questions Raised • Strengths • Looking Beyond Procedural Knowledge • Task Design • Questioning • Attending to the Responses of Children • Areas for Growth • Transfer: Moving between individual and whole class analysis of understanding • Interpreting Responses of Children • Issues of Equity

22. Using the Video Case Method for Programmatic Assessment of a Secondary Teacher Education Program Jenny Conner, Ph.D. Paula Magee, Ph.D.

23. Overview of Secondary Ed. Program • Semester 1 • Educational Psychology • Multicultural Education • Special Education • Semester 2 • Middle School Methods • Content Area Literacy

24. Overview of Secondary Ed. Program • Semester 3 • Special Methods Course (i.e., English/Language Arts Methods) • High School Methods • Semester 4 • Student Teaching

25. End of Semester 3 Benchmark III Placement • Semester 3 • Special Methods Course (i.e., English/Language Arts Methods) • High School Methods • Semester 4 • Student Teaching

26. Benchmark Testing Conditions • Computer lab • CDs with video clip • Computer software • Media Player (or other program to play media files) • Microsoft Word • Headphones

27. Benchmark III in the Secondary Ed. Program • Principles of Teacher Education (PTE’s) aligned with Principles of Undergraduate Learning (PUL’s) • Following on the heels of the elementary education programmatic assessment • Assessing PTE 1 and PTE 3…?

28. Why the Video Case Method? • Standardize the experience • Testing conditions • Time • Video clip • Reduce scoring time for faculty

29. Field-Testing • 10 English/Language Arts pre-service teachers from current semester three • Paid $50 each • 90 minutes for the task • 16 ½ minute video clip • Participants answered feedback questions

30. Feedback Question Results • Needed 2 hours • Audio/video was easy to see/hear • Did not like content specific questions • “I think it is unfair to assess someone on if they can come up with 2 books that would work with this lesson and an activity without allowing them time to research and really find a couple of good fits…”

31. Feedback Question Results • Overall felt that the task would allow students to demonstrate knowledge gained during the 3 semesters • “It asked questions that I would not have been able to answer before I entered the blocks [semesters] and took my content methods.”

32. Deeper Look at the Questions • General vs. Specific • Reasons for our decision • Concerns about our questions • Too many embedded in each • Leading • Will they work for all content areas? • Do they get at “Teaching for Understanding”?

33. Question #6 and PTE 3 Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.” No student responses alluded to “goals and choices.”

34. Question #6 and PTE 3 Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.”

35. Question #6 and PTE 3 Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.”

36. Question #6 and PTE 3 Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.”

37. Question #6 and PTE 3 Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.”

38. Question 6: Not a lot of depth Q6: ”Comment on the overall quality of the lesson you observed. What were its strengths? It’s weaknesses? Why? Be specific.” • “Overall, I think that the lesson was effective and that students were engaged in higher-order thinking.” • “The classroom here provides a forum that fosters learning and dialogue – a very important thing for middle school.” • “The lesson was decent but all of the questions asked above addressed the issues of the lesson, so I feel this question could be answered in more detail with the help from all of the questions above.”

39. Next Steps • Rubric • Developing for other content areas: • Social studies • Math • Science • Foreign Language • Art • P.E. • Integrative Departmental Grant