Kyunghwa Lee, ESSE Moonyoung Eom , LEAP Keon-ryeong Park, ESSE

1. Student Achievement in the Era of a Global Economy: Unpacking Political Discourses on International Comparisons and TeacherQuality Kyunghwa Lee, ESSE MoonyoungEom, LEAP Keon-ryeong Park, ESSE

2. International Comparisons in Political Rhetoric Secretary Arne Duncan's Remarks We continually see evidence that our children aren't getting that great education…. Compared to their peers in other countries, our students are stagnating. In science, our eighth graders are behind their peers in eight countries that also participated in the original international assessment. In math, although scores have improved somewhat since 1995, our 15 year-olds' scores now lag behind those of 31 countries. Four countries—Korea, Singapore, Hong Kong and Finland—outperform U.S. students on math, science and all other subjects. (Remarks to the President's Council of Advisors on Science and Technology, October 23, 2009; Retrieved from http://www.ed.gov/news/speeches/secretary-arne-duncans-remarks-presidents-council-advisors-science-and-technology) … countries like Singapore, South Korea, and the Czech Republic that outperform us in science and math provide teachers with much clearer guidance on key ideas and content to be mastered in each grade. (Remarks at Teachers College, Columbia University, October 22, 2009; Retrieved from http://www.ed.gov/news/speeches/teacher-preparation-reforming-uncertain-profession)

3. International test results show that we are 10th in the world in 8th grade science and fifth in the world in 8th grade math. Part of the problem is that many of our teachers in these subjects in the middle grades don't have the content knowledge. Another part of the problem is that many education schools are not preparing teachers well for the classroom…the education crisis now threatens America's ability to protect itself—let alone our ability to compete in the global economy…. Educators also must learn to use student achievement data—first to improve instruction by identifying the needs of our students -- second to insure that teachers who need more help get it -- and finally to identify teachers who can't measure up -- even with help. Linking student data to teachers, and teachers to their schools of education, can be a powerful tool in driving improvement. Why are such basic systems so rare? (Remarks at the U.S. Chamber of Commerce's Education and Workforce SummitNovember 9, 2009; Retrieved fromhttp://www.ed.gov/news/speeches/economic-security-and-21st-century-education)

4. Purpose • To question assumptions in political rhetoric linking international comparative studies on student achievement to teacher quality by examining two large cross-national studies, TIMSS and PISA • To caution educators and policy makers against developing educational practices and policies based on simplistic cross-national comparisons

5. TIMSS What does TIMSS stand for? Trends in International Mathematics and Science Study How has the study been conducted? - a major source for international comparative information on the mathematics and science achievement of students in the fourth and eighth grades and on related contextual aspects such as mathematics and science curricula and classroom practices across countries. - sponsored by the International Association for the Evaluation of Educational Achievement (IEA). - Begun in 1995, TIMSS is administered every four years (1995, 1999, 2003, and 2007). -Thirty-six countries at 4th grade and 48 countries at 8th grade participated in TIMSS 2007, including 17 countries that are members of the OECD. Source: http://nces.ed.gov/timss/index.asp

6. TIMSS Scores of 8th Grade Students(U.S./Korea/Top 3 Countries)

7. PISA • What does PISA stand for? Program for International Student Assessment • How has the study been conducted? • Program for International Student Assessment (PISA) is a system of international assessments that focus on 15-year-olds' capabilities in reading literacy, mathematics literacy, and science literacy. • PISA is organized by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries. • Begun in 2000, PISA is administered every 3 years (2000, 2003, 2006, & 2009). Each administration includes assessments of all three subjects (reading literacy, mathematics literacy, and science literacy), but assesses one of the subjects in depth. The most recent administration was in 2009 and focused on reading literacy (2000: Reading/2003: Math/2006: Science). Source: http://nces.ed.gov/surveys/pisa/

8. PISA Trends for Student Achievement Source: Lemke, M. et al. (2001), Lemke, M. et al. (2004), & Baldi, S., Jin, Y., Skemer, M., Green, P.J., and Herget, D. (2007). * PISA 2006 reading literacy results are not reported for the United States because of an error in printing the test booklets (Baldi, S. et al., 2007, p. 3).

9. Questions about Teachers in TIMSS & PISA See the attached document for the comparison between TIMSS and PISA questionnaires.

10. National Level of Teacher Quality (TIMSS 2003) • In the United States, 95.4% of 8th graders are taught by fully certified teachers, • In Korea, 99.1% of 8th graders are taught by fully certified teachers. • Higher than the international average of 91.2%. (Source) Akiba, M., LeTendre, G. K., & Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369-387.

11. Opportunity Gap: the difference btw students of high & low SES in access to qualified teachers in TIMSS 2003 • In the United States, 67.6% of high-SES students were taught by qualified teachers (e.g. with full certification, mathematics or mathematics education major, and at least 3 years of teaching experience) compared with 53.2% of low-SES students, showing the opportunity gap of 14.4%.This is significantly larger than the international average of 2.5% and fourth highest among 46 countries. • Korea showed the opportunity gap of -3.9%. (Source) Akiba, M., LeTendre, G. K., & Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369-387.

12. Akiba et al. (2007) pointed out that South Korea had achieved “both a high percentage of students taught by qualified teachers and equality of access to qualified teachers by high-SES and low-SES students” (p. 380). However, in Korea, the achievement gap between high-SES and low-SES students was 109, the same as the U.S., higher than the cross-national average of 85.8.  Low opportunity gap and high achievement gap As Akiba et al.’s (2007) data showed, “the opportunity gap in students’ access to qualified teachers was not significantly associated with achievement gap between high-SES and low-SES students” (p. 381). Examples from Korea:Low opportunity gap & High achievement gap(TIMSS 2003 data, as cited in Akiba et al., 2007) Source: Akiba, M., LeTendre, G. K., & Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369-387.

13. PISA on Teacher Quality • Table. Principal’s Perceptions of Teachers’ Morale and Commitment in PISA • There is a common belief that teachers’ morale and commitment are important factors to consist of teacher quality and to affect student achievement. • According to the results of PISA 2000 and 2003, U.S. teachers’ morale and commitment is much higher than Korean teachers’ (OECD, 2004). • Given Korean teachers’ lower morale and commitment, what makes Korean students’ achievement higher than US students’? OECD Avg. above / OECD Avg. below Source: OECD (2004). Learning for tomorrow’s world: First results from PISA 2003, p. 413.

14. Invisible/Neglected Cultural Factors • Table. • Student Participation in Educational Courses Outside School • Examples from Korea • PISA 2000 – participation courses outside school in Korea Source: OECD (2001). Knowledge and skills for life: First results from PISA 2000, p. 301.

15. Invisible/Neglected Cultural Factors(cont.) • Table. • Hours Spent “out-of school” Activities Source: OECD (2004). Learning for tomorrow’s world: First results from PISA 2003, p. 431. • PISA 2003 – hours spent outside school in Korea

16. Invisible/Neglected Cultural Factors(cont.) • Examples from Korea: The impact of private tutoring • Since 2007, the Korean National Statistical Office has started actual condition survey on private tutoring of K-12 grades twice per year (June/October). • This survey is reflecting the serious impacts of private education on educational opportunity and outcome gap in Korea (Kang, 2008). • Many studies also report that high student achievement level and gap of Korea are attributed to not school effects, including teacher quality, but excessive private tutoring (Baker, et al., 2002; Kang, 2008; Kim, 2009; Ryu, 2006)

17. Invisible/Neglected Cultural Factors(cont.) • Table. • Student Participation Rates & Hours Spent on Private Tutoring per Week Source: Statistics Korea (2008; 2009; 2010). • According to Kim (2009), the weight of private tutoring on total amount of family consumption increased from 9.4 % (2003) to 14.0 % (2008) and the weight on family’s total educational expenditure also increased from 43.9 % (2003) to 65 % (2008). • Total amounts of private tutoring in Korea was estimated at 20 billion as of 2007. This amount accounted for 2.8 % of total GDP and 65 % of total budget of the national education government body as of 2007 (Kim, 2009). • Examples from Korea: The impact of private tutoring

18. Conclusions • Linking each country’s student achievement to its teacher quality is misleading because of • Inconsistent and insufficient questions related to teacher quality in the research instruments • Various neglected/invisible cultural factors (e.g., private tutoring outside the school) affecting student achievement

19. Implications Can the comparative “underachievement” …in the United States then be reduced, without undue sacrifice, by changing the relevant sociocultural conditions of the United States to approximate those of the Asian countries?... Because every culture has a more or less coherent matrix of values, policy-oriented researchers…must consider carefully whether an intended change can be induced in the given matrix at all and whether doing so will do harm to the matrix’s coherence. (Hatano & Inagaki, 1998)

20. Example: Parent Involvement Last November, during his trip to Asia, President Obama sat down to a working lunch with South Korean President Lee in Seoul. In the space of little more than a generation, South Korea had developed one of the world's best-educated workforces and fastest-growing economies. President Obama was curious about how South Korea had done it. So he asked President Lee, "What is the biggest education challenge you have?” Without hesitating, President Lee replied, "The biggest challenge I have is that my parents are too demanding.” That anecdote usually makes American chuckle. It also makes them wince a little bit. I wish my biggest challenge—that America's biggest educational challenge—was too many parents demanding academic rigor. I wish parents were beating down my doors, demanding a better education for their children, now! President Lee, by the way, wasn't trying to rib President Obama. He explained to President Obama that his biggest problem was that Korean parents, even his poorest families, were insisting on importing thousands of English teachers so their children could learn English in first grade—instead of having to wait until second grade. That is the reality of what U.S. schoolchildren are competing against in the global economy. And it is a reality that many parents, lawmakers, and voters in America still have not yet fully grasped. In practical terms, globalization means that U.S. students will have to compete throughout their careers with their peers in South Korea, Canada, China, European countries, India, and other rapidly developing states. This race to boost educational attainment and economic competitiveness is a race that—to be brutally honest—the United States is losing. (Secretary Duncan's Remarks to the Council on Foreign Relations, New York City, October 19, 2010; Retrieved from http://www.ed.gov/news/speeches/education-and-international-competition-win-win-game-secretary-duncans-remarks-council)

21. References Baker, D. P., Goesling, B., & Letendre, G. K. (2002). Socioeconomic status, school quality, and national economic development: A cross-national analysis of the "heyneman-loxley effect" on mathematics and science achievement. Comparative Education Review, 46(3), 291-312. Baldi, S., Jin, Y., Skemer, M., Green, P.J., & Herget, D. (2007). Highlights from PISA 2006: Performance of U.S. 15-year-old students in science and mathematics literacy in an international context . Washington, DC: U.S. Department of Education, National Center for Education Statistics. Hatano, G., & Inagaki, K. (1998). Cultural contexts of schooling revisited: A review of the learning gap from a cultural psychology perspective. In S. G. Paris & M. W. Henry (Eds). Global prospects for education (pp. 79-104). Washington DC: American Psychological Association. Kang, Y. (2008). Actual condition and effective plans for dealing with private education. Seoul, Korea: Korean Educational Development Institute. Kim, M. R. (2009). Korean household expenditure on private tutoring: Its value and tobit analysis. The Journal of Economics and Finance of Education, 18(2), 1-25. Lemke, M. et al. (2001). Outcomes of learning: Results from the 2000 Program for International Student Assessment of 15-year-olds in reading, mathematics, and science literacy. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

22. References Lemke, M. et al. (2004). International outcomes of learning in mathematics literacy and problem solving: PISA 2003 results from the U.S. perspective. Washington, DC: U.S. Department of Education, National Center for Education Statistics. OECD (2001). Knowledge and skills for life: First results from PISA 2000. Paris: Organizational for Economic Co-operation and Development, Centre for Educational Research and Innovation. OECD (2004). Learning for tomorrow’s world: First results from PISA 2003. Paris: Organizational for Economic Co-operation and Development, Centre for Educational Research and Innovation. Ryu, H. G. (2006). Social stratum and educational gap. Seoul: Korea Research Institute for vocational education and training. Statistics Korea (2010). The press release of actual condition survey 2009 on private education in Korea. Statistics Korea (2009). The press release of actual condition survey 2008 on private education in Korea. Statistics Korea (2008). The press release of actual condition survey 2007 on private education in Korea.