1 / 67

ICCS 2009 Design, Databases and Materials

ICCS 2009 Design, Databases and Materials. Content. Survey Design Status of Preparation International Database Structure: File Types, Countries, Variables, Missing Data Documentation: Codebooks, Instruments, IDB User Guide Seminar Material Analysis (Do‘s and Dont‘s). Content.

skyler
Download Presentation

ICCS 2009 Design, Databases and Materials

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICCS 2009Design, Databases and Materials

  2. Content • Survey Design • Status of Preparation • International Database • Structure: File Types, Countries, Variables, Missing Data • Documentation: Codebooks, Instruments, IDB User Guide • Seminar Material • Analysis (Do‘s and Dont‘s)

  3. Content • Survey Design • Status of Preparation • International Database • Structure: File Types, Countries, Variables, Missing Data • Documentation: Codebooks, Instruments, IDB User Guide • Seminar Material • Analysis (Do‘s and Dont‘s)

  4. Survey Design – Objective • ICCS 2009 is intended to investigate the ways in which young people are prepared to undertake their roles as citizens. • It will report on: • Student achievement (civic knowledge) • Student disposition and attitudes • Techers‘ teaching and class management practices • School governance and climate • National context of civic and citizenship education • Asian, European and Latin American regional issues

  5. Survey Design – Surveys • Student Survey • Student Achievement Booklets • Student Questionnaire • Teacher Survey • Teacher Questionnaire • School Survey • School Questionnaire • National Context Survey • National Context Questionnaire

  6. Survey Design – Target Populations • Students: All students enrolled in the grade that represents eight years of schooling, counting from the first year of ISCED Level 1, providing the mean age at the time of testing is at least 13.5 years (usually grade 8). Students older than 17 years are not part of the target population. • Teachers: All teachers teaching regular school subjects to students of the target grade (regardless of the subject or the number of hours taught) during the ICCS testing period, and have been employed at school since the beginning of the school year.

  7. Survey Design – Participating Countries • 38 countries from around the world Source: Schulz et al., 2010, ICCS 2009: International Report, IEA, Amsterdam

  8. Survey Design – Sample Structure • Stratified multi-stage probability sampling design • Random samples were required for each population • Minimum sample sizes: • 150 schools • 1 intact class per school, aiming for an achieved sample size of 3,000 target grade students • 15 target grade teachers (or all if number of target grade teachers was less or equal to 20)

  9. Survey Design – Sample Structure • More than 5.300 schools, 140.000 target grade students, and 62.000 target grade teachers

  10. Survey Design – Sample Structure • Reporting of data from the student survey need to be cautious for some countries and reflect the following issues: • HKG and NLD did not meet sampling requirements. • BFL, CZE, DNK, NZL, NOR, CHE, THA met guidelines for sampling participation rates only after replacement schools were included. • ENG nearly satisfied guidelines for sample participation only after replacement schools were included. • GTM, KOR, PRY surveyed the same cohort of students but at the beginning of the next school year. • SVK national desired population does not cover all of international desired population.

  11. Survey Design – Sample Structure • Reporting of data from the teacher survey need to be cautious for some countries and reflect the following issues: • AUT, BFL, DEN, ENG, HKG, LUX, NZL, NOR, and CHE did not meet sampling requirements. • CZE, SWE, THA met guidelines for sampling participation rates only after replacement schools were included. • IRL nearly satisfied guidelines for sample participation only after replacement schools were included. • SVK national desired population does not cover all of international desired population. • GRC and NLD data from the teacher survey could not be reported. • All the above is annotated in the international reports.

  12. Survey Design – Sample Structure • More information about the sampling design will be detailed in chapters 2 and 10 of the ICCS 2009 Technical Report

  13. Survey Design – CIVED Link • Data from 17 countries could be linked to data from the IEA CIVED study from 1999. • To cover a similar target population, four countries surveyed an additional grade (grade 9): GRC, NOR, SWE, SVN

  14. Content • Survey Design • Status of Preparation • International Database • Structure: File Types, Countries, Variables, Missing Data • Documentation: Codebooks, Instruments, IDB User Guide • Seminar Material • Analysis (Do‘s and Dont‘s)

  15. Status of Preparation • Data itself confirmed by countries • Data Processing at the IEA DPC • Data preparation done • Derived variables added • Weights added

  16. Status of Preparation • Changes since last data send out (v3.1, April 2010) • Current data version 3.2 • Scale SCSTUDOP (Principals' perceptions of student opportunities to participate in community activities) in School data file (ICG) changed so that category “not offered at school” is now coded “0” instead of receiving a missing code • 3 new scales added to Student Questionnaire data file (ISG): • NISB: National Index of Socioeconomic Background • PAREDYRS: Highest educational level in approximate years of schooling • EXPEDYRS: Expected years of schooling (recoded from SISCED)

  17. Status of Preparation • Timeline for future data send out • (November 23 sent out to NRCs) • January 2011: put on IEA website for public download, including Technical Report and IDB User Guide

  18. Content • Survey Design • Status of Preparation • International Database • Structure: File Types, Countries, Variables, Missing Data • Documentation: Codebooks, Instruments, IDB User Guide • Seminar Material • Analysis (Do‘s and Dont‘s)

  19. International Database - Content • Country specific files • Data files (labeled SPSS format, SAS format, national variables where applicable) • Documentation • Codebooks (WinDEM format) • Technical Report • IDB User Guide • Data and documentation will be published in January 2011 on IEA website: www.iea.nl 19

  20. International Database - Files Types ICG School Questionnaire File ITG Teacher Questionnaire File ISG Student Questionnaire File ISA Student Achievement File ISR Student Reliability File ISE European Module Student File ISL Latin American Module Student File ISS Asian Module Student File 20

  21. International Database - Files Types Additional grade data (GRC, NOR, SWE, SVN) JSG Student Questionnaire File JSA Student Achievement File JSR Student Reliability File JSE European Module Student File 21

  22. International Database - Files Types • Cases included: • School questionnaires that were returned(CPART = 3) • Teacher questionnaires that were returned (TPART = 3)AND where schools met the minimum within school teacher participation rate of 50% • Student data that were returned (SPART = 3)AND where schools met the minimum within school student participation rate of 50%NOTE: A student was counted participating if either achievement or questionnaire data was available (regional module data only was not sufficient) • Indicator variable: INICS09 = 1

  23. International Database - Files Types • Note that the student files and also the teacher file may contain cases from schools where the school questionnaire has not been returned.

  24. International Database - Data Structure • From the survey • ID variables (record, scorer, puncher, linking) and, for students, booklet rotation • Listing and tracking variables (e.g., ITPARTR) • Response variables and, for students, constructed response scores 24

  25. International Database - Data Structure • From the data processing and weighting stage • Participation (SPART, TPART, CPART) and adjudication flags (INICS09) • Stratification information (IDSTRATE/I) • Design and estimation weights • Variables needed for variance estimation (JKZONES, JKREPS) • Meta data (VERSION, IDCNTRY, DPCDATE) 25

  26. International Database - National Adaptations • With the information in the NAFs national adaptations could be reflected and data were recoded to fit international database structure in the majority of cases • Full back-translation into English will be important for readers of the report and also for secondary research (especially if more than only country specific terms < > were adapted) 26

  27. International Database - National Adaptations • Main goal: document the comparability or non-comparability of country data • Documentation of important cultural and other national adaptations that affect the meaning of a variable 27

  28. International Database - National Adaptations • If national adaptations made recoding of data necessary, this has been documented in the National Adaptations Reports • If international comparability is not achievable, data for international variable(s) were set to not administered (but country data were still available as national variable) • If possible, data for not administered variable(s) were gang punched to a certain value • E.g., ITLANG has been gang punched to ‘1’ if only one language was used for all instruments in a country 28

  29. International Database - National Adaptations • National Adaptations Reports will inform and support secondary analysis • Will be published as part of the ICCS 2009 complementary publication (IDB User Guide) 29

  30. International Database - Variables • Identification variables (ID…) • Tracking variables (IT…) • Survey instrument variables • Weighting variables • Scale scores

  31. Identification Variables • Unique Identifiers • IDSCHOOL • School ID • 4 digits • Used in all files

  32. Identification Variables • IDCLASS • Class ID • 6 digits • First 4 digits identify school • Last two digits used to identify class within school • Used in the student files (ISG, ISA, ISR, ISE, ISL, ISS)

  33. Identification Variables • IDTEACH • Teacher ID • 6 digits • Used in teacher file (ITG) • IMPORTANT NOTE: Teachers are NOT linked to classes and NOT directly linked to students!

  34. Identification Variables • IDSTUD • Student ID • 8 digits • First 6 digits include the corresponding class (IDCLASS) • Used in student data files

  35. Tracking Variables • CPARTTPARTSPART  Final Participation Indicators • ITLANG Administration Language (1, 2, ..., 5) • IDCNTRY Country ID for Reporting • VERSION Data Sendout Version (currently 32) • DPCDATE  Date Exported (currently 20101116)

  36. Survey Variables • IC2G01A…IC2G23  survey instrument variables in school data file (ICG) • IT2G01A... IT2G29I  survey instrument variables in teacher data file (ITG)

  37. Survey Variables • CI2COM1…CI106M1  survey instrument variables in student achievement data file (ISA) • IS2G03... IS2G18G  student background variables in student questionnaire data file (ISG) • IS2P19A... IS2P36F  student perception variables in student questionnaire data file (ISG)

  38. Survey Variables • EST01A…EST12  student achievement variables in European module data file (ISE) • ESP01A…ESP13D  student perception variables in European module data file (ISE) • LST01…LST16  student achievement variables in Latin American module data file (ISL) • LSP01A…LSP12I  student perception variables in Latin American module data file (ISL) • ASP01A…ASP08E  student perception variables in Asian module data file (ISS)

  39. Survey Variables • All survey instrument variables that have been adapted for national purposes, have been recoded to fit the international scheme • National variables will be provided to country, but not be part of international database

  40. Variables for Weighting and VE School data file • TOTWGTC Final school weight – school study • JKZONEC  Jackknife zones – school study • JKREPC  Jackknife replicate code – school study

  41. Variables for Weighting and VE Teacher data file • TOTWGTT Final teacher weight – teacher study • JKZONET  Jackknife zones – teacher study • JKREPT  Jackknife replicate code – teacher study

  42. Variables for Weighting and VE Student data files • TOTWGTS Final student weight – student study • JKZONES  Jackknife zones – student study • JKREPS  Jackknife replicate code – student study

  43. Content Knowledge Variables • PV1CIV…PV5CIV: Civic knowledge plausible values • Centered to mean of 500 and standard deviation of 100 scale points (for equally weighted countries) • NWLCIV: National civic knowledge score (WLE)

  44. Scales & Derived Variables • For derived indices, see the list of indices(C:\ICCS2009\Documentation\ICCS2009_Indices.xls) • Example: RELIG • Students’ belonging to a religion • Dummy coded to ‘No religion’ (code 0) – ‘Religion’ (code 1)

  45. Scales & Derived Variables • For derived scales, see the scale documentation(C:\ICCS2009\Documentation\ICCS2009_Scales.doc) • Example: POLDISC • Students' discussion of political and social issues outside of school • IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries • Items included • IS2G13A • IS2G13D • IS2G13F • IS2G13G

  46. Missing Values • Special codes represent missing data • Codes contain information on nature of missing data • Codes and levels of missing data should be checked before usage

  47. Missing Values – “Omitted” (SPSS)‏ 9 99 ... No Response

  48. Missing Values – “Invalid” (SPSS)‏ 7 97 ... Invalid Response

  49. Missing Values – “Not Reached” (SPSS)‏ 2 (SPSS)‏ 9 99 … Question omitted (SPSS)‏ 6 96 … Question not reached

  50. Missing Values – “Not Administered” • Nationally not administered variables International (SPSS)‏ System-missing . National

More Related