190 likes | 321 Views
Reporting International Test Data Ch. 3 Measuring Civic Knowledge and Understanding. ICCS NRC meeting Madrid, Feb 2010. Measuring Civic and Citizenship Knowledge. Instrument Described scale Results. Measuring Civic and Citizenship Knowledge - Instrument.
E N D
Reporting International Test DataCh. 3 Measuring Civic Knowledge and Understanding ICCS NRC meeting Madrid, Feb 2010
Measuring Civic and Citizenship Knowledge • Instrument • Described scale • Results
Measuring Civic and Citizenship Knowledge - Instrument • 80 item (reduced to 79 - CI2HRM2 removed) • 73 MCQ, 6 constructed response • 62 new, 17 CIVED trend • Items showing extreme item by country interaction (>1.5 logits) removed from scaling for estimates of test scores in individual countries • Report to include example release items illustrating range of item types and content (including % correct by country and scale location)
Measuring Civic and Citizenship Knowledge – Described Scale • rp = 0.62 (relative shift item difficulties = ln(0.62/0.36) ≈0.49 logits
Measuring Civic and Citizenship Knowledge – Described Scale • Item map (ordered item descriptors)
Measuring Civic and Citizenship Knowledge – Described Scale • Item map (ordered item descriptors) L3 563 0.8 logits L2 479 L1 395 B1
Measuring Civic and Citizenship Knowledge – Described Scale 16 Items 39 Items 22 Items 2 Items
Measuring Civic and Citizenship Knowledge – Described Scale Level 3: holistic; strategic Evaluation Level 2: interconnectedness of systems; wider sphere of citizen influence Specificity Level 1: broad concepts; “big ideas”; mechanistic
Measuring Civic and Citizenship Knowledge – Results Comparison of means • Means range from 380 to 576 • Levels 1 to 3
Measuring Civic and Citizenship Knowledge – Results Comparison of means Multiple comparison
Measuring Civic and Citizenship Knowledge – Results • Comparison of levels
Measuring Civic and Citizenship Knowledge – Results • Comparison of levels
Measuring Civic and Citizenship Knowledge – Results • Comparison of levels
Measuring Civic and Citizenship Knowledge – Results Gender differences in achievement • International means • Girls 511 • Boys 489 • Difference is statistically significant • Mean for girls higher in all countries • Statistically significantly higher in 32 countries • Range of mean differences 2 to 48 (median =22)
Measuring Civic and Citizenship Knowledge – Results Trend Comparisons • Calculated only on the basis of 17 trend items (different metric to ICCS) • Dimensionality conflating results for complete scaling from both ends • CIVED through representativeness of trend items • ICCS through broadening of framework and constructs
Measuring Civic and Citizenship Knowledge – Results Comparisons over time • 17 countries used same translations of CIVED items for comparison • CIVED population mean = 500 • ICCS population mean = 484 • Difference is statistically significant 5 countries (stat sig one country) 11 countries (stat sig six countries)