1 / 28

The small schools dilemma – reliable analysis and target setting from NAPLAN data

The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational Measurement and School Accountability Directorate (EMSAD). Small school data analysis.

uta
Download Presentation

The small schools dilemma – reliable analysis and target setting from NAPLAN data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational Measurement and School Accountability Directorate (EMSAD)

  2. Small school data analysis Medium and large school analysis relies on comparison of outcomes with those of previous years and with outcomes of similar schools. Small schools analysis needs to focus on the progress of cohorts rather than comparison of outcomes for different cohorts.

  3. Data instability • Aggregated data for small schools are inherently unstable. • Confidence intervals around means and band percentages are much greater than for larger schools. • Cohort differences are likely to be much greater.

  4. Percentage in bands – large school

  5. Means trend – large school

  6. Percentage in bands – small school

  7. Means trend – small school

  8. Small school confidence intervals Confidence interval of +/- 52

  9. Large school confidence intervals Confidence interval of +/- 13

  10. What is a ‘small school’ for analysis and reporting purposes • Reporting limits are usually set out of privacy considerations. ACARA uses a minimum of 5 students. NSW DEC uses a minimum of 10 students. • Numbers required for comparison of cohort performance are greater than for reporting purposes. • Overall school performance measures used in school performance graphs require results for a minimum of 15 students to construct a ‘reliable’ estimate. (Performance measures based on 10 to 14 students are provided but should be interpreted with caution.) • The decisions you make regarding analysis and target setting need to be based on the size of the group and the nature of the data (e.g. means or band percentages.)

  11. School size data The average enrolment of NSW DEC primary schools is 260 students (approximately 37 per year). Ten per cent of NSW DEC primary schools have less than 23 students (3 per year). Twenty per cent of NSW DEC primary schools have less than 50 students (7 per year) Thirty per cent of NSW DEC primary schools have less than 90 students (13 per year) About a third of NSW DEC primary schools could described as ‘Small schools’ for analysis and reporting purposes.

  12. Percentage of government primary schools with less than 50 students

  13. Useful data sources for small schools • Expected growth data • Item analyses by syllabus outcomes for tracked cohorts and individual students • Relative performance graphs • Data aggregated over several years

  14. Y3 to Y5 average growth in reading by Y3 reading score

  15. Expected growth • Expected growth differs according to the starting score. • Approximately 57% of Year 5 students achieved expected growth in reading

  16. Year 5 student growth chart

  17. Year 7 student growth chart

  18. Cohort item analysis – Year 3 reading 2008 RS 2.5: Makes some inferences about ideas implicit in texts

  19. Cohort item analysis – Year 5 reading 2010 RS 2.5: Makes some inferences about ideas implicit in texts

  20. Individual item analysis – Year 3 reading 2008 RS 2.5: Makes some inferences about ideas implicit in texts

  21. Relative performance – numeracy v reading

  22. Relative performance – writing v reading

  23. Using rolling 3-Year average data

  24. Target setting for small schools • Targets should be achievable (realistic) and (preferably) measurable. • It is preferable if small school targets don’t rely on comparison with raw performance of previous cohorts. Comparison with 3-year averages may be appropriate. • Small school targets are best set in relation to the prior performance of the students to whom the targets relate.

  25. Target setting options • Using the school target setting tool (Replace Year 5 data with Year 3 data and express targets in terms of percentage proficient, for example. ) • Expected growth targets • Three year rolling average targets • Improvement targets in terms of syllabus outcomes (data extracted from individual student learning plans).

  26. Using the school target setting tool Replace Year 5 data with 2009 Year 3 band percentages

  27. Possible types of targets • Increase percentage of all students with individual learning plans (K-6) achieving 80% of their learning goals for reading (from 2011 achievement average of 66%) • Increase the percentage of all students achieving greater than or equal to expected growth to 70% in 2011 (from 59% in 2010) • Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from X% to Y% in Year 5 2011 • Increase the three year aggregated percentage of Year 5 students achieving in the top three bands to 81% (from the 2008-2010 average of 87.5%).

More Related