Data Management Procedures

1. Data Management Procedures Michael Jung, IEA Data Processing and Research Center

2. Contents • Receipt of FT Data and Documentation • FT Data Processing at the IEA DPC • Evaluation of FT Data Quality • WinW3S - Within-School Sampling Software • WinDEM – Data Entry Manager • SurveySystem – On-line Data Collection Software • Material Submission after Test Administration • MS Data Processing at the IEA DPC

3. Receipt of FT Data and Documentation

4. Receipt of FT Data and Documentation • Data from 32 countries altogether • 16 data sets arrived on time • 16 data sets arrived after January, 25th

5. Receipt of FT Data 25-Jan

6. Receipt of FT Data and Documentation • Data from 32 countries altogether • 16 data sets arrived on time • 16 data sets arrived after January, 25th • Documentation from 32 countries altogether • 10 countries sent on time and complete • 22 countries sent after January, 25th and complete

7. Receipt of FT Documentation 25-Jan

8. FT Data Processing at the IEA DPC

9. Data & Codebooks IEA DPC Country Instruments & Forms Data Processing at the IEA DPC 1. Import to SAS and adapting the file structure INPUT

10. Adapting the File Structure • WinW3S database and WinDEM data files have been merged, renamed and converted to SAS data files • Unique system numbers (CSYSTEM, TSYSTEM, SSYSTEM) have been added • National Scores and derived variables (e.g. final participation indicator) have been added • National variables have been removed (will be sent to countries in a separate data file on demand)

11. Structure of the Merged Data Set Example: Student Questionnaire Data File Information: Tracking Forms Information: Student Questionnaire Added during cleaning process WinDEM (ISGxxxF2.DBF)

12. Data & Codebooks Country Database IEA DPC Country Instruments & Forms New Data Structure Reports, Statistics & Docu. Data Processing at the IEA DPC Structure Check Background Cleaning INPUT OUTPUT 2. Cleaning ID Cleaning Linkage Cleaning

13. Evaluation of FT Data Quality

14. Evaluation of FT Data Quality • General data quality in almost all countries was VERY GOOD • Most common problems • Not all changes in the instruments reflected in the codebooks • Not administered variables deleted in codebooks instead of hidden • WinDEM quality control checks not performed or not all problems have been solved • Inconsistencies between Student Questionnaire files and Occupation data files • Information in WinW3S not completed and thus database not exported

15. WinW3S - Within-School Sampling Software

16. WinW3S • Management tool for all ICCS within-school sampling operations • Identifies the schools, classes, teachers and students that will participate in ICCS • Creates a hierarchical ID system + passwords for ODC • Samples classes + teachers • Assigns booklets to the students • Produces and prints all forms and labels

17. WinW3S – What's New • New option for participation in Asian Module added • Import of students and teachers from MS Excel files • New participation code for students added: „participated, but excluded” • New participation code for teachers added: „teacher on temporary leave” • Assignment of reliability booklets changed: random assignment of ~100 R booklets per booklet type • Assignment of different class IDs for ICCS Target Grade and Additional Grade students • Listing of teachers birth information and gender optional on Teacher Listing Forms • Possible to list more than 99 teachers per school • Teacher and school principal questionnaire mode can be changed on individual level • ODC optional ONLY for countries that have trialed ODC in FT • Participation status for on-line participants can be imported from SurveySystem Monitor

18. WinDEM – Data Entry Manager

19. WinDEM • Provides mask for data entry from booklets and paper questionnaires • Based on codebooks, provided by the IEA DPC • Allows adaptation of codebooks • Validates data during data entry • Provides various data verification checks after data entry • Allows for comparison of double-entered data

20. WinDEM – What's New • New coding module for coding of occupations integrated in WinDEM • Entry of occupation data directly from WinDEM • Occupation data will be stored in an MS Access database • Text answers and occupation codes also available in table mode

21. IEA SurveySystem On-line Data Collection Software

22. ODC – What's New • Export tool in SurveySystem Monitor application for participation status of teachers and school principals • data can be imported into WinW3S later on

23. Material Submission after Test Administration

24. WinW3S database exported for IEA DPC Sampling • (MICS<country>1.dbf) • (MICS<country>2.dbf) • National Adaptation Forms (IV) (*.xls) • Set of final National Instruments (*.pdf) Material to be submitted Send Material to IEA DPC Sampling Send Material to ISC at ACER Send Material to IEA DPC

25. National Adaptation Forms (IV) (*.xls) • Set of final National Instruments (*.pdf) • WinW3S database exported for • IEA DPC • (MICS<country>1.mdb) • (MICS<country>2.mdb) • Student and Teacher Tracking Forms (paper copies or scanned) Send Material to IEA DPC Sampling Send Material to ISC at ACER Material to be submitted Send Material to IEA DPC

26. Material to be submitted • WinDEM data files (*.dbf) + • WinDEM codebooks (*.sdb) • (ICGICSM2.DBF; *SDB) • (ITGICSM2.DBF; *SDB) • (ISGICSM2.DBF; *SDB) • (ISAICSM2.DBF; *SDB) • (ISRICSM2.DBF; *SDB) • (ISEICSM2.DBF; *SDB) • (ISLICSM2.DBF; *SDB) • (ISSICSM2.DBF; *SDB) • Occupation data file • (ISOICSM2.mdb) Send Material to IEA DPC Sampling Send Material to ISC at ACER Send Material to IEA DPC

27. IEA DPC Sampling ISC at ACER IEA DPC Material to be submitted Data and Documents have to be submitted by 27-Mar-2009 latest for SH countries and by 29-Aug-2009 latest for NH countries 

28. File-Exchange using the IEA DPC WebFTP-Server • The IEA DPC has setup secure FTP sites to exchange materials between countries and the - ISC at ACER and/ or - IEA Secretariat and/ or - IEA DPC • Can easily be accessed using a browser (e.g. Internet Explorer) • Individual country logins from FT are still valid • FT folders and documents have been moved to a folder “0_Old_Material_from_Field_Trial “  please do not upload any documents into that folder any more

29. File-Exchange using the IEA DPC WebFTP-Server • MS Materials should be uploaded using a predefined folder structure 1_Instruments_and_NAF_for_Adaptation_Verification_ACER 2_Instruments_and_NAF_for_Translation_Verification_IEA_Secretariat 3_Instruments_and_NAF_for_Layout_Verification_ACER 4_Final_Instruments_and_NAF_for_ACER _and_IEA_DPC 5_SurveySystem_Definition_Files_for_IEA_DPC 6_Codebooks_and_Data_Files_for_IEA_DPC 7_Exported_WinW3S_Database_for_IEA_DPC 8_Other_Materials 9_To_Country

30. File-Exchange using the IEA DPC WebFTP-Server • Please inform the recipients via email as soon you have uploaded any materials • Updated WebFTP-Server Manual will be sent to countries

31. MS Data Processing at the IEA DPC

32. Data Processing at the IEA DPC • Data Processing of the MS data at the IEA DPC basically follows the same prcedures as for the FT • But: Any problems and/or inconsitencies will be reported back to countries and need correction and/or approval • Countries need to be available after data submission

33. Data & Codebooks Country Database IEA DPC Country Instruments & Forms New Data Structure Reports, Statistics & Docu. Data Processing at the IEA DPC Structure Check Background Cleaning INPUT OUTPUT Cleaning ID Cleaning Linkage Cleaning

34. Thank you very much for your attention.