490 likes | 593 Views
Universiteit van Amsterdam AMSTEL Instituut. Benefits and limitations of international comparative research in science and mathematics education. AMSTEL – Seminar Pauline Vos. International comparison.
E N D
Universiteit van Amsterdam AMSTEL Instituut Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos 22 April 2009
International comparison • Use identical instruments in different countries and look at differences between countries. • My own international experience: • Went to school in NL, D and F • worked in NL, Zimbabwe, Mozambique & South Africa • Worked at the Dutch research centre for TIMSS-1999 (University of Twente) • PhD on mathematics curriculum in NL using TIMSS-instruments
Many variations • Bilateral studies • Affect in science: ROSE (Relevance of Science Education), 40 countries • Teacher competencies: TEDS(M), 17 countries
Two abbreviations… • PISA = Project for International Student Assessmentorganised by OECD(Organisation for Economic Cooperation and Development) • TIMSS = Trends in International Maths and Science Studypreviously: Third Internat. Maths & Science Studyorganised by IEA (Internat. Assoc. for the Evaluation of Educational Achievement)
Translate and define • Mathematics • Basic calculations and further • Science • physics, chemistry, biology, physical geography, astronomy, ecology, etc • Students • all students (not just pre-university)
Some history… • 1964 A comparison of 12 countries, Husén & Bloom • 1971 FISS • 1980-82 SIMS • 1984 SISS • 1995 TIMSS (later known as “TIMSS-1995” ) • 1999 TIMSS-Repeat (later known as “TIMSS-1999”) • 2003 TIMSS-2003 • 2007 TIMSS-2007 • 2008 TIMSS Advanced TIMSS-2011 • 2000 PISA-2000 • 2003 PISA-2003 • 2006 PISA-2006 PISA-2009
How can you do research on the question: To what extent do countries differ in the education of science and mathematics? 22 April 2009
Both studies focus on a test 22 April 2009
Complex studies .. • Assessment framework • Large number of items • Long design process for instruments • Large random sample • Quality control • trend items (retained) • coding and re-coding • excluding countries when criteria were not met
Complex studies… (1) • Not just measuring country scores in one discipline, but also: • Sub-disciplines (algebra, ecology, ..) • Sub-competencies (reproduce, explain, communicate, ..) • Background variables • Not just measuring students’ scores, but also trying to explain: • Differences between countries • Differences between schools • Differences between students
Complex studies… (2) • PISA: 85 exercises distributed over 13 test booklets • PISA-2000 • Main topic: Literacy (reading) • Additional: science, maths • PISA-2003: • Main topic: Mathematical Literacy • Additional: literacy, science, problem solving • PISA-2006 • Main topic: Science • Additional: literacy, maths
Complex studies… (3) • TIMSS: 300 exercises distributed over 12 booklets • TIMSS-95: • grades 3, 4, grades 7, 8 • Performance Assessment (= test in a lab-like environment) • grade 12: math/science literacy; advanced maths/science • video study (USA, Jap, Germ.) • TIMSS-99: • grade 8 • TIMSS-R Video study (USA, Jap, NL, Cz, Switz, Austr, HongK.) • TIMSS-2003 • grade 4, grade 8 • TIMSS-2007 • grade 4, grade 8 • TIMSS-2008 • grade 12
Math Item Formats *PISA definitions: Multiple Choice includes regular and “complex” multiple-choice items. Short Constructed Response includes “short answer” and “closed response” items. Extended Constructed Response includes “open response” items.
Horse race - thinkingboth in PISA and TIMSS • Ranking is simplistic • Sample: intrinsically inaccurate • Scores of some countries are very close. With s.e.>3 are small differences not significant • NL position in 2003: • math: on shared positions 2-7 in PISA and shared positions 6-9 in TIMSS • science: on shared positions 4-8 in PISA and shared positions 7-11 in TIMSS
Differences PISA and TIMSS • Population: • PISA: age cohort: 15-year olds (indep. of grade) • TIMSS: grade (indep. of age) • Countries: • PISA: economically developed (OECD) • TIMSS: whole world (incl. Saudi Arabia, Botswana) • Cyclus: • PISA: 3-yearly with different thematic foci • TIMSS: 4-yearly with different populations • Content: • PISA: applicable knowledge (situations): reading, math. literacy, scientific literacy • TIMSS: school knowledge (US curriculum): mathematics, science • Justification of the study: • PISA: knowledge has economical importance • TIMSS: knowledge about education is needed
Grade 8 seven countries: Australia, Hong Kong, Japan, NL, Czech Rep, USA, Switzerland Aim: describe patterns in lessons question: what is typical for the math lessons in a certain country? TIMSS-1999 Video Study • in each country 85 random lesson filmed, transcribed & coded • Points of analysis: • Structure of the lesson: e.g. who speaks? How many exercises? • Mathematical content: complexity, procedures • Char of exercise: symbols, contexts
TIMSS-Video study: use of materials: • Percentage lessons in which a calculator is used
TIMSS-Video study: structure of lessons • Percentage lessons in which a goal is stated
TIMSS-Video study: structure of lessons • Percentage lessons, in which a summary statement is made
TIMSS-Video study: exercises • Percentage exercises per lesson with a context of symbols only
USA video USA Grade 8 (14 yrs) Heterogeneous non-problem segment:the teacher shows the class examples of angles formed by secants and tangents intersecting inside, on, or outside a circle. TIMSS-1999 Video Study
Netherlands video NL Grade 8 (14 yrs) Homogeneous (pre-university) non-problem segment:the teacher briefly presents historical background related to the Pythagorean theorem (i.e., Pythagoras was a Greek gentleman). She also connects ideas learned previously (i.e., the definitions of various types of triangles) to the present lesson. TIMSS-1999 Video Study
Hong Kong: grd 8, heterogeneous Explanation of a new concept. video HongKong Pre-knowledge: students can solve linear equations What preceded the scene: Solving2x+10=2(x+5)2x+10=2x+10 0 = 0 Then different values were insterted for x . left hand sideright hand side“expanded form”“factorized form” TIMSS-1999 Video Study
Next in this theatre… • TEDS(M) report in December 2009 • TIMSS-2008 Advanced report in December 2009 • Pisa 2009 report in 2010
References for PISA and TIMSS • www.pisa.oecd.org • http://timss.bc.edu
remains the question…. what are the benefits and limitations?