250 likes | 331 Views
Study Overview. National Research Coordinators Meeting Amsterdam, October 16-19. IEA studies in the field. 1971: First IEA Civic Education Study in 10 countries (part of six-subject study) 1999/2000: IEA CIVED Assessment of 14-year-olds in 28 countries
E N D
Study Overview National Research Coordinators Meeting Amsterdam, October 16-19
IEA studies in the field • 1971: First IEA Civic Education Study in 10 countries (part of six-subject study) • 1999/2000: IEA CIVED • Assessment of 14-year-olds in 28 countries • Assessment of upper secondary students in 16 countries • 2008/2009: IEA ICCS (more than 40 expressions of interest so far)
International Study Centre and partner institutions - 1 • Australian Council for Educational Research (ACER), Melbourne/Australia • International Study Centre • Overall Project management • Test Development • Student Background Questionnaire • Scaling and Data analysis • John Ainley (Project Coordinator) • Wolfram Schulz (Research Director)
International Study Centre and partner institutions - 2 • National Foundation for Educational Research (NFER), Slough/UK • National Contexts Survey • Student Perceptions Questionnaire • European Regional Module • David Kerr (Associate Research Director)
International Study Centre and partner institutions - 3 • Laboratorio di Pedagogia Sperimentale (LPS), Rome/Italy • School Questionnaire • Teacher Questionnaire • Bruno Losito (Associate Research Director)
International Study Centre and partner institutions - 4 • IEA Data Processing Center (DPC), Hamburg/Germany • Sampling and Weighting • Data Processing • Scaling and Analysis • Falk Brese (Coordinator) • Heiko Sibberns (TEG and PAC representative)
International Study Centre and partner institutions - 5 • IEA Secretariat, Amsterdam/Netherlands • Translation Verification • Quality Monitoring • Barbara Malak (Manager Membership Relations) • Hans Wagemaker (Executive Director)
Project Management • Joint Management Committee (JMC) • Meets frequently (≥ monthly) with consortium staff • Meets occasionally as a JMC • Project Advisory Committee (PAC) • 1st meeting July 2006 • 2nd meeting October 2006 • IEA Secretariat • IEA Technical Expert Group (TEG)
Purposes of study • International comparison of outcomes of civic and citizenship education at start of 21st century • Provide trend data for comparison with CIVED 1999 • Response to challenge in civic and citizenship education in changed contexts of democracy and participation
ICCS and CIVED • Opportunity to review changes • Extended framework and broadening of scope • Different approach to teacher survey • Regional modules
Research questions directed at exploration of • Variations in student achievement • Changes since 1999 • Extent of interest and dispositions towards citizenship • Student perceptions of recent threats and future developments • Relationship of learning outcomes with curriculum, teaching and school organisation • Relationship of learning outcomes with student background
Assessment Framework • Outlines what will be assessed in ICCS • Consists of two parts: • Civics and Citizenship framework • Contextual framework • Current version to be discussed at this meeting
Instrumentation • Student instruments comprising: • International (core) cognitive test (60 minutes) • International (core) student background questions (15 minutes) • International (core) student perceptions on civic-related issues (30 minutes) • Regional cognitive test module(s) (20-30 minutes) • Regional student questionnaire on civic-related issues (10-20 minutes) • Teacher questionnaire (30 minutes) • School questionnaire (30 minutes) • National Contexts Survey (on-line)
Instrument development • Establishing assessment framework • Piloting of items in subset of participating countries (Feb-Mar 07) • Review with experts and NRCs • Field Trial of preliminary set of instruments (Oct-Nov 07) • Review with experts and NRCs • Final selection (Feb-Apr 08)
General survey design • Population: Grade 8 (minimum average age of 13.5) • Two-stage cluster samples in participating countries • PPS sampling of (at least 150) schools • Random selection of one classroom within each sampled school • Random selection of teachers within sampled schools (no student-teacher link) teaching grade 8
Instrument administration • Test administrators responsible for test and questionnaire sessions • Student instrument administration: • International cognitive test • International background questionnaire • International perceptions questionnaire • Regional Cognitive test (optional) • Regional perceptions questionnaire (optional)
Translation and verification • Instruments translated from (English) source version into national languages • National adaptations reviewed in cooperation with international study centre • Translations verified by language experts • Final layout review
Field Operations • Manuals will be developed to guide operations undertaken by national centres (coordination at school, test administration, coding, data entry) • School and teacher questionnaires administered through school coordinators • International control monitors visiting sub-samples of schools
Data Management and Scaling • DPC creates international database (cleaning procedures) • DPC computes sampling weights • DPC provides item statistics • DPC and ACER undertake scaling of data • IRT scaling model still to be decided (1-PL vs. 3-PL) • Plausible values for cognitive measures • IRT weighted likelihood scores for some questionnaire-based indices
Analysis and Reporting • (Probably) Jackknife Repeated Replication (JRR) to provide correct standard errors • Described scales (cognitive and some affective-behavioural) • Descriptive tables with uni- and bivariate analyses • Single and multi-level analyses
Reports • International Report (June 2010) • Regional Report(s) (June 2010) • Technical Report (Nov 2010) • Instrument development • Field Operations • Sampling • Scaling • Reporting
International Database • Student, teacher and school databases including: • ID variables • Item responses • Cognitive Scale(s) – Plausible Values • Affective-Behavioural Scales • Weights • JRR Indicators
Assistance to NRCs • Item development workshop (this meeting) • Data Management and Marker Training seminars (FT and MS) • International Database Seminar (Nov 2010)