Ambitious Instructional Practices in Mathematics: Do They Matter in Outcomes?

1. Ambitious Instructional Practices in Mathematics: Do They Matter in Outcomes? Ali Korkmaz, Ph.D. AhmetUludag, Ph.D. NiluferKorkmaz, Ph.D. Accord Institute for Education Research

2. What Do We Know? Background Engagement Educational Performance Long-term Social Engagement Student characteristics Demographics Attitudes Educational background Educational Attainment Retention Academic Engagement Academic Achievement Family-School-Community, Resources Rumberger and Larson’s Framework for Studying Student Educational Performance

3. Student Achievement

4. Theoretical Background • Walberg’s Educational Productivity Model (1984, 1992) • Student aptitude • ability or prior achievement • development • motivation, or self-concept • Instruction • the amount of time students engage in learning • the quality of the instructional experience • The environment factors encompass four items: • the home • the classroom social group • the peer group outside the school • use of out-of-school time

5. Background • Marzano’s (2000) effective instructional strategies (teacher-level factors) • Instruction • Classroom management • Curriculum design • Hattie’s Visible Learning (2008) • Meta analysis of instructional practices effects on student outcomes

6. Teachers • Clear learning intentions • Challenging success criteria • Range of learning strategies • Know when students are not progressing • Providing feedback • Visibly learns themselves

7. Students • Understand learning intentions • Are challenged by success criteria • Develop a range of learning strategies • Know when they are not progressing • Seek feedback • Visibly teach themselves

8. Purpose of the study • What types of instructional practices are implemented in classrooms? • What types of instructional practices are the game changers in math? • What practices lead to more growth among non-proficient students? • What practices lead to more growth among proficient students?

9. Study Design • Student Survey • Instructional practices • Study habits, motivation • Home environment • School learning environment Student Fall Math Test Student Spring Math Test

10. Study Design Elements • Northwest Evaluation Association (NWEA) MAP Test • Consortium on Chicago School Research (CCSR) Student Survey • Longitudinal growth for mathematics

11. NWEA MAP Tests A computer adaptive test tool • to measure student levels, placement and differentiate instruction to meet student needs • to guide curriculum and instructional decisions • to measure student growth over time • of accountability to see how well we have done our jobs; to measure the effectiveness of curriculum and instruction

12. Target Growth = 5 Spring Target RIT 215 Fall RIT 210 Spring Target RIT 202 Target Growth = 10 192 Fall RIT Target Growth for two 4th Grade Students

13. RIT scores – Growth - Targets • RIT: Rasch (Rash) Unit is a scale that shows student levels independent of grade level and age • RIT scores are used to show a student’s level normative to other students and their grade level through percentile ranks • Growth Targets: Typical growth is given for grade levels, yet each student has a separate growth target based on their initial score • Two things to look at (at the end of the year): • How many students have met their target • How many points have the students grown on average (compared to typical growth)

14. Characteristics of the RIT Scale : • It is an achievement scale. • It is an equal interval scale. • It helps to measure growth over time. • It has the same meaning regardless of grade or age of the student.

15. Proficiency vs. Growth

16. Math Growth from Fall to: (2011 Norm)

17. Student Survey • Consortium on Chicago School Research (CCSR) Surveys • More than 10 years in the field • IRT based (Rasch) survey items and constructs • It is designed mainly for school level analysis • Student survey items include: • Academic Engagement, Study Habits, Classroom Behavior, Learning Climate (Expectation, Relationships) • Subject area specific questions • School and home environment questions

18. Ambitious Instruction In schools with strong Ambitious Instruction, classes are challenging and engaging. The instruction is clear, well-structured, and encourages students to build and apply knowledge. When combined with a supportive environment, Ambitious Instruction has the most direct effect on student learning. It is: • well-defined with clear expectations for student success, • interactive and encourages students to build and apply knowledge, • well-paced (not measured), and • aligned across grades (not measured).

19. Reliability for Scales

20. Data • 846 students at 9 CA middle schools • 55% male • 50% free-reduced lunch • 11% African-American • 45% White • 44% Hispanic

21. Data Analyses • Students are grouped into 6 groups based on Fall proficiency and Fall to Spring Growth • Survey scales compared between groups with effect-size

22. Data Cleaning • Survey time less than 10 minutes • MAP test time less than 10 minutes

23. Results Teacher Personal Attention (11 items) (strongly agree – strongly disagree) • Notices if I have trouble learning something • Will help me improve my work if I do poorly on an assignment • Is willing to give extra help on schoolwork if I need it • Helps me catch up if I am behind • Gives me specific suggestions about how I can improve my work in this class • Explains things a different way if I don't understand something in class High Growth – Non Proficient > High Decline – Non Proficient +.20

24. Results Course Clarity (5 items) (strongly agree – strongly disagree) • I learn a lot from feedback on my work. • The textbook and other course materials my teacher gives me really helps me understand the topics we are studying. • It is clear what I need to do to get a good grade. • The work we do in class is good preparation for the test. • The homework assignments help me learn the course material. High Growth – Non Proficient > High Decline – Non Proficient +.29 High Growth –Proficient > High Decline – Proficient +.32 HG–Prof > HG– Non Prof > HD – Prof > HD - NP

25. Results Academic Engagement (10 items) (strongly agree – strongly disagree) • The topics we are studying are interesting and challenging. • You have to come to class every day to keep up. • I am usually bored in class. • I usually look forward to this class. • I work hard to do my best in this class. High Growth – Non Proficient > High Decline – Non Proficient +.27 High Growth –Proficient > High Decline – Proficient +.31 HG–Prof > HG– Non Prof > HD – Prof > HD - NP

26. Results Quality of Student Discussion (4 items) (A Great Deal – Very Little) • We build off each others' ideas. • We talk about different solutions or points of view. • Our discussions connect what we're learning to things we've studied in the past. • If someone makes an incorrect statement, it gets corrected. High Growth – Non Proficient > High Decline – Non Proficient +.25

27. Results Quality Math Instruction (5 items) (Almost every day – Never) • Write a few sentences to explain how you solved a math problem • Explain how you solved a problem to the class • Write a math problem for other students to solve • Discuss possible solutions to problems with other students • Apply math to situations in life outside of school High Growth – Non Proficient > High Decline – Non Proficient +.20 High Growth –Proficient > High Decline – Proficient +.28 HG–Prof > HG– Non Prof > HD – Prof > HD - NP

28. Results Rigorous Study Habits (11 items) (strongly agree – strongly disagree) • I set aside time to do my homework and study. • I try to do well on my school work even when it isn't interesting to me. • If I need to study, I don't go out with my friends. • I finish whatever I begin. • I manage my time well enough to get all my work done. High Growth –Proficient > High Decline – Proficient +.41 HG–Prof > HG– Non Prof > HD – Prof = HD - NP

29. Results Student-Teacher Trust (6 items) (strongly agree – strongly disagree) • My teachers always keep his/her promises. • My teachers always try to be fair. • My teachers will always listen to student's ideas. • My teachers treat me with respect. High Growth –Proficient > High Decline – Proficient +.49

30. Results Incidence of Disciplinary Action (4 items) (More than 5 times – Never) • You gotten into trouble at school? • You been sent to the office for getting into trouble? • You been suspended from school? • Your parents been contacted because you got into trouble? High Growth –Proficient < High Decline – Proficient -.60

31. Results Struggling Environment (3 items) (strongly agree – strongly disagree) • This has been a difficult year for my family. • I have had problems this year with kids at school or in my neighborhood. • I have many personal responsibilities outside of school. High Growth – Non Proficient < High Decline – Non Proficient -.20 High Growth –Proficient < High Decline – Proficient -.39 HG–Prof < HG– Non Prof < HD – Prof < HD - NP

32. Summary

33. Implications • Use of effective student surveys • Inform what is happening • Student and classroom level analysis • For professional development plans • For policy • Teacher evaluations?

34. Limitations • Student survey • Teacher input on classroom practices (teacher survey) • Student growth • High stakes, any rewards etc. involved

35. Further Study • Classroom level analysis of the same data set • What types of practices work better at what types of classrooms? • Combine student surveys with teacher surveys

37. Contact Ali Korkmaz, Ph.D. akorkmaz@accordeducation.org AhmetUludag, Ph.D. auludag@accordeducation.org CERA - November 29, 2012