1 / 62

Evaluation Institute Qatar Comprehensive Educational Assessment (QCEA) 2008 Summary of Results

Summary of the QCEA 2008 program results, including independent school data, gender disparities, and subject-specific performance levels. Comparison with previous years and cohort analysis is also discussed.

epropp
Download Presentation

Evaluation Institute Qatar Comprehensive Educational Assessment (QCEA) 2008 Summary of Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation InstituteQatar Comprehensive Educational Assessment (QCEA) 2008Summary of Results

  2. Report Outline • Overview of the QCEA program • Better understanding the 2008 QCEA results • QCEA 2008 independent school results, by grade • Key assumptions • Summary of results • Arabic: overall, gender, and independent school cohort • English: overall, gender, and independent school cohort • Mathematics: overall, gender, and independent school cohort • Science: overall, gender, and independent school cohort • QCEA 2005, 2006, 2007, and 2008 comparisons ofindependent school results across time • Key assumptions • Summary of results • Arabic: All students and independent school cohort • English: All students and independent school cohort • Mathematics: All students and independent school cohort • Science: All students and independent school cohort • QCEA 2005 student cohorts: selected results within subject areas and across time • Key assumptions • Summary of results • Student cohort performance-level changes: Arabic, English, mathematics and Science • Student cohort performance-level changes by gender: Arabic, English, mathematics and science • Tracking and comparing aspects of student cohort performance: Arabic and English 2

  3. Overview of the QCEA program QCEA program • Measures student performance with respect to the Qatar Curriculum Standards • Assesses Arabic, English, mathematics, and science learning in grades 4 to11 • Administered to approximately 28,000 students, attending Qatar independent schools in the Spring of 2008 About reporting the 2008 QCEA results • Individual Student Reports distributed to schools and Family Reports distributed to parents and students • Class Reports by subject-matter for teachers contain data about their current students 3

  4. Better understanding the QCEA 2008 results Performance levels and scale scores: • Each student receives a scale score in each content area and scores in some sub-areas. • Scale scores are also converted into performance levels, indicating whether a student has met the standards. • Five performance levels are measured, with three embedded within the “Below Standards” level to better discriminate among the poorer performers: • “Meets Standards” • “Approaches Standards” • “Below Standards”: from Level 1 (lowest) to Level 3 (highest) 4

  5. QCEA 2008 independent school results by grade Key assumptions Summary of results Arabic English Mathematics Science } overall, gender, and independent school cohort

  6. Key Assumptions: QCEA 2008Performance level results across subject-matter: overall, gender, and cohort QCEA 2008 results include any student with: • a valid score • attending any independent school that year • attending grades 4–11 Overall results include: • approximately 26,000 students with valid results in each subject-matter Results by gender include approximately: • 14,000 Boys • 12,000 Girls Results by cohort include approximately: • 6,500 school Cohort 1 students • 8,000 school Cohort 2 students • 5,000 school Cohort 3 students • 6,500 school Cohort 4 students 6

  7. Summary of Results: QCEA 2008Overview Overall, 2008 performance levels are generally low. Some students perform at the very highest levels, but the majority are performing at moderate levels, at best. Performance in English is marginally better than that in the other three subject areas tested. For all grades, with isolated exceptions, low proportions of students performed at the “Meets Standards” level (typically about 10% of students in English, about 5% or less of students in Arabic and close to 0% of the students in mathematics and science). The longer term picture (examining the results from 2004 to 2008) is no brighter: there is some variability in performance for different grades, subject areas, and years, however, there are no clear-cut indications of improvement over time. 7

  8. Summary of Results: QCEA 2008Performance level results across subject-matter: overall and gender Subject-matter • Although scale score comparisons across subject-matters are not valid, performance level proportions can be contrasted. In this respect, students tended to perform better on the Arabic and English tests than on the mathematics and science instruments. • A larger proportion of students scored in the “Meets Standards” performance level in Arabic and English, as compared to mathematics and science. • A smaller proportion of students scored in the “Below Standards: Level 1” in Arabic and English, as compared to mathematics and science. Gender • Girls tended to perform better than boys in Arabic, English, and science • However, in mathematics: • Boys had a higher percentage of scores in the combined “Meets Standards” and “Approaches Standards” performance levels in four grades, and girls had a higher percentage in the other four grades. • Girls had a smaller percentage of scores in the “Below Standards: Level 1” in all grades. 8

  9. Summary of Results: QCEA 2008Performance level results across subject-matter: independent school cohort Independent school cohort • Across all grades and subject-matter, Cohort 1 tended to perform better than Cohort 2; and both Cohort 1 and Cohort 2 tended to perform better than Cohort 3 and Cohort 4. Exceptions include: • English, grade 8, where Cohort 3 performed slightly better than Cohort 2 • Mathematics, grade 8, where Cohort 3 performed slightly better than Cohort 2 • Science, grades 4, 5, 6, 8, and 9 where Cohort 4 performed better than Cohort 2 • Science, grades 4, 5, 6, and 9 where Cohort 4 performed better than, or similar to, Cohort 1 9

  10. Arabic: overall Performance level results, QCEA 2008 • By grade • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 8% (grade 5) • Approaches Standards ranges from 25% (grade 8) to 32% (grade 4) • Below Standards ranges from 62% (grade 5) to 73% (grade 9) 10

  11. Arabic: genderPerformance level results, QCEA 2008 • Boys • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 5% (grade 7) • Approaches Standards ranges from 17% (grade 10) to 25% (grade 7) • Below Standards ranges from 70% (grade 7) to 81% (grade 10) • Girls • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 12% (grade 11) • Approaches Standards ranges from 31% (grade 8) to 39% (grade 11) • Below Standards ranges from 49% (grade 11) to 67% (grade 9) 11

  12. Arabic: school cohortPerformance level results, QCEA 2008 • Cohort 1 • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 15% (grade 11) • Approaches Standards ranges from 36%(grade 8) to 43% (grade 11) • Cohort 2 • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 7% (grade 11) • Approaches Standards ranges from 25% (grade 8) to 35% (grade 7) • Cohort 3 • Percentage of students in: • Meets Standards ranges from 1% (grades 4 and 9) to 5% (grade 5) • Approaches Standards ranges from 11% (grade 11) to 30% (grade 7) • Cohort 4 • Percentage of students in: • Meets Standards ranges from 1% (grade 9) to 5% (grade 5) • Approaches Standards ranges from 17% (grades 7 to 9) to 24% (grades 4 and 6 ) 12

  13. English: overallPerformance level results, QCEA 2008 • By grade • Percentage of students in: • Meets Standards ranges from 7% (grade 8) to 13% (grade 5) • Approaches Standards ranges from 11% (grade 9) to 34% (grades 5 and 6) • Below Standards ranges from 53% (grade 5) to 81% (grade 9) 13

  14. English:gender Performance level results, QCEA 2008 • Boys • Percentage of students in: • Meets Standards ranges from 5% (grades 9) to 8% (grades 4to 6) • Approaches Standards ranges from 9% (grade 9) to 29% (grade 5) • Below Standards ranges from 63% (grade 5) to 86% (grade 9) • Girls • Percentage of students in: • Meets Standards ranges from 10% (grades 8 and 9) to 17% (grade 5) • Approaches Standards ranges from 14% (grade 9) to 42% (grade 6) • Below Standards ranges from 43% (grade 6) to 75% (grade 9) 14

  15. English: school cohortPerformance level results, QCEA 2008 • Cohort 1 • Percentage of students in: • Meets Standards ranges from 17% (grade 8) to 30% (grade 11) • Approaches Standards ranges from 24% (grade 9) to 47% (grades 4 and 5) • Cohort 2 • Percentage of students in: • Meets Standards ranges from 4% (grade 8) to 11% (grade 5) • Approaches Standards ranges from 9% (grade 9) to 37% (grades 4 and 6) • Cohort 3 • Percentage of students in: • Meets Standards ranges from 2% (grades 4 to 6) to 9% (grades 7 and 8) • Approaches Standards ranges from 8% (grade 10) to 23% (grade 8) • Cohort 4 • Percentage of students in: • Meets Standards ranges from Less than 1% (grade 9) to 6% (grade 4) • Approaches Standards ranges from 4% (grade 9) to 30% (grade 6) 15

  16. Mathematics:overallPerformance level results, QCEA 2008 • By grade • Percentage of students in: • Meets Standards ranges fromless than 1% (grades 4 to 9) to 5% (grade 10) • Approaches Standards ranges from 15% (grade 4) to 46% (grade 10) • Below Standards ranges from 49% (grade 10) to 85% (grade 4) 16

  17. Mathematics: gender Performance level results, QCEA 2008 • Boys • Percentage of students in: • Meets Standards ranges from 0% (grade 4) to 5% (grade 10) • Approaches Standards ranges from 14% (grade 11) to 38% • (grade 10) • Below Standards ranges from 57% (grade 10) to 86% (grade 11) • Girls • Percentage of students in: • Meets Standards ranges from 0% (grade 7) to 5% (grade 10) • Approaches Standards ranges from 14% (grades 4 and 9) to 56% (grade 10) • Below Standards ranges from 39% (grade 10) to 86% (grades 4 and 9) 17

  18. Mathematics: school cohortPerformance level results, QCEA 2008 • Cohort 1 • Percentage of students in: • Meets Standards ranges from less than 1% (grades 4 and 6) to 14% (grade 10) • Approaches Standards ranges from 21% (grade 4) to 53% (grade 6) • Cohort 2 • Percentage of students in: • Meets Standards ranges from less than 1% (grades 4 to 9 and 11) to 5% (grade 10) • Approaches Standards ranges from 12% (grade 4 and 9) to 46% (grade 10) • Cohort 3 • Percentage of students in: • Meets Standards ranges from 0% (grades 4 to 6, 9 and 11) to 3% (grade10) • Approaches Standards ranges from 5% (grade 4) to 46% (grade 10) • Cohort 4 • Percentage of students in: • Meets Standards ranges from 0% (grades 4 to 6, 9 and 11) to 1% (grade 10) • Approaches Standards ranges from 4% (grade 9) to 44% (grade 10) 18

  19. Science: overallPerformance level results, QCEA 2008 • By grade • Percentage of students in: • Meets Standardsranges from 0% (grades 9 to 11) to less than 1% (grades 4 to 8) • Approaches Standards ranges from 18% (grade 7) to 35% (grade 9) • Below Standards ranges from 65% (grade 9) to 82% (grade 7) 19

  20. Science: genderPerformance level results, QCEA 2008 • Boys • Percentage of students in: • Meets Standards ranges from 0% (grades 7 and 9 to 11) to less than 1% (grades 4 to 6 and 8) • Approaches Standards ranges from 16% (grade 7) to 35% (grade 9) • Below Standards ranges from 65% (grade 9) to 84% (grade 7) • Girls • Percentage of students in: • Meets Standards ranges from 0% (grades 9 to 11) to 1% (grade 8) • Approaches Standards ranges from 20% (grade 7) to 39% (grade 11) • Below Standards ranges from 61% (grade 11) to 80% (grade 7) 20

  21. Science: school cohortPerformance level results, QCEA 2008 • Cohort 1 • Percentage of students in: • Meets Standards ranges from 0% (grades 4 and 9 to 11) to 1% (grade 8) • Approaches Standards ranges from 20% (grade 5) to 46% (grade 10) • Cohort 2 • Percentage of students in: • Meets Standards ranges from 0% (grades 9 to 11) to 1% (grade 4) • Approaches Standards ranges from 13% (grade 7) to 29% (grade 9) • Cohort 3 • Percentage of students in: • Meets Standards ranges from 0% (grades 4 to 7 and 9 to 11) to less than 1% (grade 8) • Approaches Standards ranges from 11% (grade 10) to 35% (grade 9) • Cohort 4 • Percentage of students in: • Meets Standards ranges from 0% (grades 6 to 11) to 1% (grade 5) • Approaches Standards ranges from 15% (grade 7) to 41% (grade 6) 21

  22. Comparisons of independent schools across time: QCEA 2005, 2006, 2007, and 2008 Key assumptions Summary of results Arabic English Mathematics Science } All students and independent school cohort

  23. Key Assumptions:QCEA 2005, 2006, 2007, 2008Interpreting trends and results for “all students” • Interpreting trends: • Trends can be examined across all years for “all students” and for school “Cohort 1” students. • For 2005, 2006, 2007, and 2008, for all grades, the “all students” group includes a large number of students who did not attend an independent school in the previous academic year. • The trends examined across years are compared across the same grade levels. • (e.g., Compare: grade 4 in 2005 to grade 4 in 2006; grade 5 in 2005 to grade 5 in 2006, etc.) • Results for “all students” includes any student with: • a valid score • enrollment in an independent school that year (all cohorts aggregated) • enrollment in grade 4 to 11 • promoted one grade each academic year “All students” includes approximately: • 4,000 students in 2005 • 12,000 students in 2006 • 17,000 students in 2007 • 26,000 students in 2008 23

  24. Key Assumptions:QCEA 2005, 2006, 2007, 2008Trends: independent school Cohort 1 • Results for independent school Cohort 1 follow the same • student cohort across grades and years. • Students are included in “Cohort 1” results if they: • fulfill the “all students” selection criteria • attended a cohort 1 school • during all four years, from 2005 to 2008, for all years the student was in grades 4 to 11, or • for fewer years if the student was in grade 4 in 2006, 2007 or 2008, or in grade 9, 10 or 11 in 2005 “Cohort 1” students includes approximately: • 2,000 students in 2005 • 2,500 students in 2006 • 3,000 students in 2007 • 4,000 students in 2008 24

  25. Summary of Results: QCEA 2005, 2006, 2007, 2008 Trends: general results General results: • “Independent school Cohort 1” results were similar to the “all students” results. However, Cohort 1 results tended to be slightly higher in most cases. • Results across years 2005, 2006, and 2007 showed a trend of decreasing or inconsistent performance. Performance tended to increase from 2007 to 2008 in most subject-matters. However, these increases often were not enough to bring the 2008 performance as high as the performance in 2005. Student cohort trends in school Cohort 1: • In most subject-matters, the grade 4 cohort’s scores in 2006 were lower than the 2005 grade 4. This student cohort trend continues across years with the 2007 grade 5 performing lower than the previous grade 5 in 2006; and the 2008 grade 6 performing lower than the previous grade 6 in 2007. • In mathematics and science the grade 7 cohort scores in 2006 were lower than the 2005 grade 7. This student cohort trend continues across years with the 2007 grade 8 performing lower than the previous grade 8 in 2006. However, the 2008 grade 9 students performed similarly to the previous grade 9 in 2007. 25

  26. Summary of Results:QCEA 2005, 2006, 2007, 2008 Trends: percentage of students in performance levels “Approaches Standards” and “Meets Standards”, and average scale score While results based on percentages of students in performance levels “Approaches” and “Meets” generally agreed with the results based on average scale scores, there were a few exceptions in the analysis including “all students”: • Increases only in average scale scores • In Arabic, students in grade 9 showed a 4-point gain in average scale score from 2005 to 2006. However, the percentage of students in performance levels “Approaches” and “Meets” decreased 11 percentage points. • In Arabic, students in grades 10 and 11 showed 14- and 12-point gains, respectively, on average scale scores from 2007 to 2008. However, the percentage of students in performance levels “Approaches” and “Meets” decreased by 1 percentage point for both grades. • Decrease only in average scale scores • In science, grade 7 students showed an 18-point decrease in average scale score from 2007 to 2008. However, the percentage of students in performance levels “Approaches” and “Meets” increased by 1 percentage point. Based on the percentages of students in performance levels “Approaches” and “Meets”: • Compared to all other grades, grade 4 tends to have the lowest percentage of students in the upper performance levels in Arabic, English, and mathematics for Cohort 1 students, and in mathematics for all students. 26

  27. Summary of Results:QCEA 2005, 2006, 2007, 2008 Trend line results: all students and independent school Cohort 1 All students: • Comparisons between 2005 and 2006 across the same grade levels: • Performance mostly decreased in Arabic, mathematics, and science. • No consistent trend was evident in English. • Comparisons between 2006 and 2007 across the same grade levels: • Performance mostly decreased in Arabic, mathematics and science. • No consistent trend was evident in English. • Comparisons between 2007 and 2008 across the same grade levels: • Performance mostly increased in English, mathematics, and science. • No consistent differences were evident in Arabic. Independent school Cohort 1 students: • Comparisons between 2005 and 2006 across the same grade levels: • Performance mostly decreased in science and Arabic. • No consistent trend was evident in English or mathematics. • Comparisons between 2006 and 2007 across the same grade levels: • Performance mostly decreased in Arabic. • No consistent trend was evident in English, mathematics, or science. • Comparisons between 2007 and 2008 across the same grade levels: • Performance mostly increased or stayed about the same in English, mathematics and science. • No consistent differences were evident in Arabic. 27

  28. Arabic: all studentsPercentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 28

  29. Arabic: independent school Cohort 1Percentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 29

  30. English: all studentsPercentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 30

  31. English: independent school Cohort 1Percentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 31

  32. Mathematics: all studentsPercentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 32

  33. Mathematics: independent school Cohort 1Percentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 33

  34. Science: all studentsPercentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 34

  35. Science: independent school Cohort 1Percentage of students in performance levels “Approaches” and “Meets”, QCEA 2005, 2006, 2007, 2008 35

  36. Arabic: all studentsAverage scale score results, QCEA 2005, 2006, 2007, 2008 36

  37. Arabic: independent school Cohort 1 Average scale score results, QCEA 2005, 2006, 2007, 2008 37

  38. English: all studentsAverage scale score results, QCEA 2005, 2006, 2007, 2008 38

  39. English: independent school Cohort 1Average scale score results, QCEA 2005, 2006, 2007, 2008 39

  40. Mathematics: all studentsAverage scale score results, QCEA 2005, 2006, 2007, 2008 40

  41. Mathematics: independent school Cohort 1 Average scale score results, QCEA 2005, 2006, 2007, 2008 41

  42. Science: all studentsAverage scale score results, QCEA 2005, 2006, 2007, 2008 42

  43. Science: independent school Cohort 1Average scale score results, QCEA 2005, 2006, 2007, 2008 43

  44. Focusing on selected aspects of the 2005 cohorts of independent school student results from 2005 to 2008 Key assumptions Summary of results Student cohort performance-level changes: Arabic, English, mathematics and science Student cohort performance-level changes by gender: Arabic, English, mathematics and science Tracking and comparing aspects of student cohort performance: Arabic and English

  45. Key Assumptions:selected 2005 student cohort performance from 2005 to 2008 • Understanding the data • Focuses on the performance of selected cohorts of students who commenced in independent schools in 2005 and continued within independent schools for all four years of schooling through to 2008 successively without repeating a grade. • This involves five cohorts: • the grade 4 2005 students who completed grade 7 in 2008 [“Grade 4 Cohort”] • the grade 5 2005 students who completed grade 8 in 2008 [“Grade 5 Cohort”] • the grade 6 2005 students who completed grade 9 in 2008 [“Grade 6 Cohort”] • the grade 7 2005 students who completed grade 10 in 2008 [“Grade 7 Cohort”] • the grade 8 2005 students who completed grade 11 in 2008 [“Grade 8 Cohort”] . • The groups are special and atypical in that they comprise students who had the advantage of having a coherent educational experience over the four years (i.e., having progressed year by year through the grades, through independent schools within similar curriculum frameworks and resourcing environments) and attended school on all testing days — these cohorts would be expected to exhibit better than ‘typical’ standards and patterns of performance and the results presented would be expected to portray a more positive picture when compared to the overall student population. • These are clearly special groups of students, but the examination of their performance as individuals over time provides for a sharper and deeper understanding of the educational progress and impacts on students who have attended independent schools. 45

  46. Key Assumptions: selected 2005 student cohort performance from 2005 to 2008 (continued) • Results for the student cohorts includes any students who: • have valid scores in each of 2005, 2006, 2007 and 2008 for the subject area(s) of focus; • were enrolled in an independent school for all four years; • started in a Cohort 1 independent school in 2005 in grade 4, 5, 6, 7 or 8; andwere promoted one grade each academic year. The student cohorts comprised approximately 2,200 students: • 400 Grade 4 Cohort students; • 560 Grade 5 Cohort students; • 410 Grade 6 Cohort students; • 380 Grade 7 Cohort students; and, • 450 Grade 8 Cohort students. 46

  47. Summary of Results:selected 2005 student cohort performance-level changes between 2005 and 2008 General performance-level change results In Arabic and mathematics, for all but the grade 4 cohort, the greatest proportion of students exhibited no change in performance-level against the standards between 2005 and 2008. The strongest performance-level change results were exhibited in English followed by Arabic, then mathematics, with science being the weakest (in science the overall net performance-level change was negative). Student cohort gender performance-level change variations In all four subject areas (Arabic, English, mathematics and science) the female student cohorts exhibit greater net performance-level improvement than the male students in the respective cohorts, except for the 2005 grade 6 cohort. On balance, it appears that the gaps which exist between female and male student performance are not closing but, if anything, widening. 47

  48. Summary of Results:selected 2005 student cohort performance-level changes between 2005 and 2008 (continued) Performance-level change considerations While it may appear that some of the improvements are good, it is important to mediate this positive outlook with the reasonable aspiration that most students should be able meet or closely approach standards. At the present rates of progress, it is unlikely that such an aspiration will be achieved by the present student cohorts. Given the relatively low performance starting points for the 2005 student cohorts, there is an inherent bias in favor of possible performance-level increases rather than performance-level decreases and the potential impact of this effect needs to be taken into account when making judgments about the magnitude of changes observed. 48

  49. Arabic:student cohort performance-level changesProportions of 2005 student cohorts whose performance-level are higher, the same, or lower in 2008 as compared to 2005 LOWER SAME HIGHER Grade 4 Cohort Grade 5 Cohort Grade 6 Cohort Grade 7 Cohort Grade 8 Cohort 49

  50. English: student cohort performance-level changesProportions of 2005 student cohorts whose performance-level are higher, the same, or lower in 2008 as compared to 2005 50

More Related