1 / 40

Using NAPLAN summative data to identify instructional successes and challenges.

Using NAPLAN summative data to identify instructional successes and challenges. . Presented by Philip Holmes-Smith School Research Evaluation and Measurement Services. Overview of Session. Using NAPLAN summative data to identify instructional successes and challenges

arleen
Download Presentation

Using NAPLAN summative data to identify instructional successes and challenges.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using NAPLAN summative data to identify instructional successes and challenges. Presented by Philip Holmes-Smith School Research Evaluation and Measurement Services

  2. Overview of Session • Using NAPLAN summative data to identify instructional successes and challenges • Understanding the National Scale, • School Summary analysis, • Five-Year Trend analysis, • Individual student-level data analysis • Value added (Growth) analysis

  3. Understanding the National Scale

  4. Understanding the National Scale • The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculumat some time in the future.

  5. Understanding the National Scale • The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculumat some time in the future. • The National Scale was fixed in 2008as follows: • Range: 0-1000 • Mean: 500 • Standard Deviation: 100 (i.e. 68% students between 400-600) 900 800 +1 STD 700 600 2008 National Mean 500 400 -1 STD 300 200 100

  6. National Averages (2008) My School Website (2010) School’s Average National Average Only 20.1% of Australian school communities are more advantaged Average for school’s with similar ICSEA

  7. Understanding the National Scale • The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculumat some time in the future. • The National Scale was fixed in 2008 as follows: • Range: 0-1000 • Mean: 500 • Standard Deviation: 100 (i.e. 68% students between 400-600) • The Scale has been divided into ten Bandswhich are used for reporting to parents. • Band 1 covers all scores equal to or less than 270. • Bands 2 – 9 increment by 52 score points each Band • Band 10 covers all scores above 686.

  8. Understanding the National Scale • The National Scale is an arbitrary scale – at this stage it is not related to points along a developmental curriculum. But, it is highly likely that it will be mapped onto the National Curriculumat some time in the future. • The National Scale was fixed in 2008 as follows: • Range: 0-1000 • Mean: 500 • Standard Deviation: 100 (i.e. 68% students between 400-600) • The Scale has been divided into ten Bandswhich are used for reporting to parents. • Band 1 covers all scores equal to or less than 270. • Bands 2 – 9 increment by 52 score points each Band • Band 10 covers all scores above 686. • At this stage Bands have no explicit curriculum meaning but results show that for Victorian students in 2011: • A typical Yr3 level of performance is at the bottom of Band 5 • A typical Yr5 level of performance is almost halfway into Band 6 • A typical Yr7 level of performance is a third into Band 7 • A typical Yr9 level of performance is at the bottom of Band 8 Yr 9 Yr 7 Yr 5 Yr 3

  9. Parent Reports(NB. Bands do not equate to Victorian Year Level means) Year 9 Year 7 Year 5 Year 3

  10. Victorian State Averages - 2011(By Year Level and Dimension)

  11. Cautionary Note #1 • Equal scores amongst different dimensions (on the National Scale) do not equate toequal levels of performance in terms of expected VELS levels. e.g. • A National Yr9 Reading score of 586.6 is equivalent to a VELS score of 5.25, but • A higher National Yr9 Numeracy score of 592.5 is equivalent to a lower VELS score of 4.82.

  12. Comparing National Scale Scores to Estimated VELS Equivalent scores • The Victorian means for Year 3 and Year 9 Reading and Numeracy on the National scale are compared to the estimated VELS equivalent scores below: • Compared to our expected curriculum outcomes for Year 3 students (2.175), the State Reading mean is about 1½ term ahead of where we expect a typical Year 3 student to be. However, the State Numeracy mean is about 2½ terms below where we expect a typical Year 3 student to be. • Compared to our expected curriculum outcomes for Year 9 students (5.175), the State Reading mean is just above where we expect a typical Year 9 student to be. However, the State Numeracy mean is about 2½ terms below where we expect a typical Year 9 student to be.

  13. Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

  14. Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

  15. Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

  16. Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

  17. Comparing AIM scores (VELS May equivalent) to NAPLAN scores (Est. VELS equivalent)

  18. Cautionary Note #2 • “Estimated VELS Equivalent Scores” should be viewed with caution. Specifically: • It is doubtful that the “Estimated VELS Equivalent Scores” for Writing,Spelling or Grammar & Punctuation are truly VELS scores. (The writing marking rubric does not use VELS criteria and there are no VELS criteria for Spelling or Grammar & Punctuation.) • “Estimated VELS Equivalent Scores” are based on an equating study that is now three years old.

  19. School Summary Analysis

  20. The School Summary Report Choose “School Summary Report” Choose to look at All Students or Girls vs. Boys or LBOTE Students or ATSI Students 2010 Choose to look at National Scale Scores (Estimated VELS scores are problematic) Click on “Preview Report” to view results

  21. The School Summary Report

  22. Interpreting “box and whisker” graphs 3 students on or above the 90th percentile. (Note, all we know is at least one student scored on the 90th percentile.) • There are 30 students in this Year Level. Therefore: • 50% (or 15 students) are above the median. • 50% (or 15 students are below the median. • 50% (or 15 students) are inside the “box”. Half of these (7-8 students) are above the median and half (7-8 students) are below the median. • 10% (or 3 students) are at or below the 10th percentile “whisker”. • 10% (or 3 students are at or above the 90th percentile “whisker”. • 15% (or 4-5 students) are spread between the 25th down to the 10th percentile. • 15% (or 4-5 students) are spread between the 75th up to the 90th percentile. ? ? 15 students above median 4-5 students between the 75th up to the 90th percentile. 7-8 spread above the median. 15 students inside the box. 7-8 bunched below the median. 4-5 students between the 25th down to the 10th percentile. 15 students below median 3 students on or below the 10th percentile. (Note, all we know is at least one student scored on the 10th percentile.) ? ?

  23. Interpreting the School Summary Report • Is the school’s median above, at or below the State median? School’s median is about half a band below the State median

  24. Interpreting the School Summary Report • Is the school’s median above, at or below the State median? • For VELS only, is the Reading median above, at or below the Numeracy median? School’s median is about half a band below the State median Reading is higher than Numeracy

  25. Interpreting the School Summary Report • Is the school’s median above, at or below the State median? • For VELS only, is the Reading median above, at or below the Numeracy median? • How does the school’s spread compare to the State spread? School’s median is about half a band below the State median Reading is higher than Numeracy School has far fewer high performers than the State School’s median is at the State’s 25th percentile School has far more low performers than the State

  26. The School Summary Report Review your own School Summary Report Go to: https://naplands.vcaa.vic.edu.au/ (Use your school’s NAPLAN Data Service logon and password to access the report)

  27. Five-Year Trend Analysis

  28. Five-Year Trend Analysis Difference scores show how much better or worse you are compared to the State average This year’s score appears to be part of an improving trend compared to the State average

  29. Five-Year Trend Analysis Look at your own SPA data Go to: http://www.sreams.com.au/ (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)

  30. Individual Student-level Analysis:Grouping Students for Teaching

  31. Individual Student-level Analysis Score that place a student one or more years below expected are coloured red Score that place a student at or about expected level are coloured yellow Score that place a student one or more years above expected are coloured green

  32. Individual Student-level Analysis Click on a dimension heading to sort the scores Group requiring more basic level material Group requiring grade level material Group requiring more advanced material

  33. Individual Student-level Analysis Look at your own SPA data Go to: http://www.sreams.com.au/ (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)

  34. Value-Added (Growth) Analysis

  35. Growth Analysis (NAPLAN scale) Letitia appears to have grown 163.6 units from 436.8 in Yr3 to 600.4 in Yr5. Javier appears to have gone backwards 40.7 units from 506.2 in Yr3 to 465.5 in Yr5.

  36. Growth Analysis (NAPLAN scale) On Average, the school has grown its students 66.79 units compared to the State’s 83.2 units. This is 16.41 units less than the state.

  37. Growth Analysis (VELS scale) Letitia appears to have grown 1.64 VELS units from 2.52 in Yr3 to 4.16 in Yr5. Javier appears to have gone backwards 0.43 VELS units from 3.11 in Yr3 to 2.68 in Yr5.

  38. Growth Analysis (VELS scale) On Average, the school has grown its students 0.68 VELS units compared to the State’s 0.84 VELS units. This is 0.16 VELS units less than the state.

  39. Growth Analysis (VELS scale) Difference scores show how much more or less growth you show compared to the State growth This year’s growth appears to be part of an worsening trend compared to the State growth

  40. Value-Added (Growth) Analysis Look at your own SPA data Go to: http://www.sreams.com.au/ (Use your SPA logon and password to access you SPA account or click on “Purchase SPA” to purchase and set up a new account for your school)

More Related