Proposed student questionnaire for the main survey

1. Proposed student questionnaire for the main survey National Research Coordinators Meeting Windsor, June 2008

2. Contents of presentation • Criteria used for selection of questionnaire material • Overview of field trial questions and recommendations • Length of proposed instrument • Proposed changes

3. Criteria for selection • Field trial analysis results • Scaling properties (for scaled items) • Item dimensionality • Face validity • Review of relationship between related variables/constructs • Coverage of assessment framework • Feasibility of adaptations to national contexts

4. General proposals • Only one (combined) student questionnaire (with both background and perceptions questions) • Only one form • No “don’t know” categories

5. Review of field trial instruments • Three categories of questions • To be retained without changes (in green) • To be modified (in yellow) • To be deleted (in red) • Some questions are flagged as having “medium priority” • Recent advice from the PAC meeting 7-8 June appears in orange

6. Student characteristics • Age (A01) • Gender (A02) • Ethnicity (optional) (A03) • Expected educational level (A04) • without “don’t know” category

7. Home environment • Household composition (B01) • Medium priority only • Country of birth (B02) • Language use at home (B03)

8. Parent background • Parental occupation (B04, B06) • Good measure of socio-economic background • Somewhat larger percentages of missing values in some countries • Parental education (B05, B07) • Without “don’t know” category • Still considerable percentages of missing values • Parental interest in political/social issues (B08) • Interesting results!

9. Household possessions • Household items (B09) • low reliabilities (especially in more developed countries) • low correlations with other socio-economic indicators) • PAC advice: Consider as an international option • Books in the home (B10) • Good indicator of socio-economic background

10. Measuring socio-economic background in the main survey • It is proposed to have three indicators • Parental occupation (highest of both parents) • Parental education (highest of both parents) • Number of books at home • Analysis will be undertaken to explore computation of composite index

11. Student activities • Out-of-school leisure time activities (C01) • Keep only items on reading and hanging out • Add: Homework and study time • PAC advice: Consider keeping also (at least) computer and television use items • Activities out of school (C02) • Discussion item form scale • Media information items do not scale but are of interest as single items (wording change for TV news) • Social participation items do not scale and should not be retained • PAC advice: Consider combining items on organised social activities (boy scouts, youth clubs etc.)

12. Students’ civic participation • Civic participation in the community (C03) • Worked as scale and interesting associations with other variables • PAC advice: An additional note to include religious services was initially considered but should rather not be included • Civic participation at school (C04) • Two items reflect different dimension (tutoring, school newspaper) and are proposed to be deleted • There were adaptation problems with the item on school protest, which is proposed as another deletion • PAC suggestion: Consider same categories as for previous question

13. School-related perceptions - 1 • Open climate for classroom discussion (D01) • Good scaling properties and interesting correlations • PAC advice: Use rather same categories as in CIVED • Student influence at school (D02) • Good scaling characteristics but negative correlations with civic knowledge • ISC sees this as question with medium priority • PAC advice: Additional item about extracurricular activities

14. School-related perceptions - 2 • School climate items (D03) • Good scaling properties for student-teacher relation items (medium priority) • Students’ sense of belonging has not always satisfactory scaling properties and should be deleted • Students’ confidence in school participation (D04) • Negatively worded items did not load on the same dimensions and should not be retained

15. Perceptions of citizens and society - 1 • Democratic value beliefs (E01) • Agreement item format leads to highly skewed responses • Alternative format proposed asked about characteristics of “ideal society” • PAC advice: Keep original format • Good citizenship beliefs (E02) • Two-dimensional structure confirmed • Proposed to delete three items that did not scale well with either of the two scales • PAC advice: Consider keeping “obeying the law” and “working hard” for single-item reporting

16. Perceptions of citizens and society - 2 • General trust (E03) • Few interesting associations • Only item about trust in “people in general” should be added to question about trust in institutions • Acceptance of socially undesirable actions (E04) • Some problems with item scaling properties and generally not too interesting results

17. Self-perceptions - 1 • Interest in political and social issue (F01) • Generally good scaling properties • Five items to be retained and one modified • Students’ self-concept (political internal efficacy) (F02) • Six items have good scaling properties • One (negatively phrased) item did not scale well and should be deleted

18. Self-perceptions - 2 • Feelings of distinctiveness (F03) • Doubts that items measure what is intended • Attachment to community levels (F04) • Results not very interesting • May be retained for regional instruments

19. Rights and responsibilities - 1 • Gender rights items (G01) • One item did not scale well and should be deleted • Rights for ethnic/racial groups (G02) • Good scaling properties • Correct adaptation need to be ensured (not minorities!)

20. Rights and responsibilities - 2 • Rights for immigrants (G03) • Negatively phrased items reflect different dimension and should not be retained • PAC suggestion: Keep item h (“When there are not many [instead of enough] jobs available the number of <immigrants> should be limited“) for single-item reporting • Reaction to threats to democracy (G04) • Items did not scale and only single-item reporting might be considered • Proposed to delete this question • PAC advice: Include three items (b, d and f) in democracy belief question (E01)

21. Perceptions of institutions and society - 1 • Trust in groups and institutions (H01) • Good scaling properties for CIVED items and interest in single-item reporting for not scaled items • Addition of “people in general” • “Standardisation” of optional items • PAC suggestion: Consider asking about media separately (Newspaper, TV, radio, internet) and ask about “people in general” first • Attitudes toward country (H02) • Generally, good scaling properties except for three items (two of them should be deleted)

22. Perceptions of institutions and society - 2 • Government responsiveness (external political efficacy) (H03) • Analysis revealed two dimensions (“responsiveness” and “cynicism”) • Both scales have weak reliabilities • Satisfaction with country’s achievements (H04) • Generally, good scaling properties • Relatively high correlations with attitudes toward country • Deletion proposed on content grounds • Mixed PAC advice

23. Perceptions of institutions and society - 3 • Support for political parties (H05) • Interesting results • Sufficient proportions in each category

24. Perceptions of participation in society - 1 • Citizenship self-efficacy (I01) • Good scaling properties for all but one item (“raising money for people in need”) • Protest activities (I02) • Two dimensional-structure (legal and illegal protest) • Shortening of legal protest scale (two items proposed for deletion)

25. Perceptions of participation in society - 2 • Participation as an adult (I03) • Two dimensions (expected electoral participation and expected active political participation) • Good scaling properties for all items • Participation as a young person (I04) • First two items reflect different dimension (community-based participation) • Good scaling properties for item measuring informal participation

26. International option regarding religion • Religious denomination (J01) • Participation in religious services (J02) • Attitudes toward religion (J03) • Good scaling properties for five items • Three items do not really reflect attitudes and should be deleted

27. Proposed student questionnaire for main survey • With 5 items per minute the estimated length is 34 minutes • There are about 170 to 180 items in the instrument • The CIVED questionnaire had 162 items in the perceptions part (for 30 minutes)

28. Proposed changes • Reduction in questionnaire length of about 10 minutes • Generally, only slight modifications or deletions are proposed • The only more fundamental change for democratic value beliefs a format change has been dropped following PAC advice • Risk of using an un-trialled question format in main survey!

29. Questions or comments?

30. Group discussions • No complete re-writing of questions possible • No additional questions • unless previously used in surveys of the same age group... • Go through the original field trial material (tables in document 6) and discuss • Agreement with proposed selection • Prioritisation of questions • Feedback on proposed • Suggestions of (minor) changes to wording

31. Group allocations • From six groups • Three groups start from the beginning of questionnaire material (those sitting in front) • Three groups start from the end (those sitting towards the back) • Try to form mixed groups (from different countries and regions)