A strategy to engage teachers in using OneSchool Data

1. A strategy to engage teachers in using OneSchool Data John Bright, Head of Department, Science, Browns Plains State High School

2. Strengths • Identify generic movements • Picks up some ultimate movement in student learning • Standardised and transparent • Clear and relatively simple reporting • Limitations • Not good as a student diagnostic • Has a performance and not developmental agenda • Not regularly conducted for students • Not sensitive for specific program evaluation (generally) • Remember! Schools should have lots of other data What are the limitations of NAPLAN?

3. This is detective work! • There are rarely easy answers. • Results can be due to many factors • (e.g.,) • Averaging maybe meaningless • Answer is often qualitative • Answer is often complex • It includes a search for patterns or lack thereof • It includes a search for corroborating evidence • It is dynamic in nature and may involve theories / models / literature Drawing accurate conclusions?

4. Colour coding in reports

5. Colour bands are broad statistical groupings, and actual scores can be noticeably below the national mean but show as blue (comparable). Comparisons are made with other students in the same demographic. Indigenous students (green shows above national mean) are performing well below non-indigenous students. Interpreting colour coding and numerical data

6. Note that even thought numeracy for 2008 and 2009 are both blue (similar to national mean), in 2008 the score is 21 points below the national figure but in 2009 7 points ahead. Comparing data from different years

7. There are 8 NAPLAN Region/District Reports (that are not available to schools) 1. All performance for school, district or region — Region 2. All performance for school, district or region — District 3. All performance for school, district or region — Schools within Region 4. All performance for school, district or region — Schools within District 5. Performance for all student demographics by strand — Region 6. Performance for all student demographics by strand — District 7. Performance for all student demographics by year level — Region 8. Performance for all student demographics by year level — District

8. Schools have access to - 2 “data exports”.- one graph - 7 School reports- 13 Student/Class reports

9. The OneSchool menu for school reports

10. School Performance data export (as Excel spreadsheet) Data rich, but beyond the comprehension of most teachers or administrators.

11. State and Like Schools data export (as Excel spreadsheet) Also data rich, but also beyond the comprehension of most teachers or administrators.

12. One Graph

13. By selecting different options you can display the data available in the data exports in a manner that is easier for many people to grasp. Graph of Mean Performance Relative to Comparison Group

14. Graph shows range of student scores and mean.

15. Data for indigenous students can be viewedremembering that the indigenous students’ data is compared with the national performance of indigenous students.

16. There are seven school reports

17. Mean scale scores by strand

18. Achievement bands by strand

19. All performance for school, district or region — Individual school

20. School 1 1 Performance measure summary – single school

21. Performance for all student demographics by strand — Individual school

22. The menu for the 13 class/cohort/individuals school reports

23. Student Summary by class/cohort — Current students Gives useful data about performance of individuals in selected subject groups – useful to individual teachers

24. Individual student item responses — Current students Gives a student’s answers to each question in a previous years test – but reasons for an incorrect answer are difficult to ascertain – one page per student.

25. Individual Student Writing Criteria students at time of test Data rich, but a teacher would need to navigate 26 pages -again, one page per student.

26. Gives all students’ answers to all question in previous years test – but reasons for an incorrect answer are difficult to ascertain. Student item response by class / cohort — Current students

27. Item level summary by class / cohort — Current students Allows teachers to obtain data about their particular class group, but, again, data rich and can be difficult for some teachers to interpret.

28. These graphs for a Year 8 Science classes performance in Yr 7 NAPLAN test, sorted by question number, demand little in the way of data skills to interpret. Item percentage correct by class / cohort – Current students

29. Writing Criteria summary – students at time of test This data can provide useful information for program writers regarding areas of weakness in the English program, particularly if patterns of poor performance repeat from year to year.

30. Relative Gain – students at time of testing. This data can only be viewed for the whole cohort, or by form class, AT THE TIME OF TESTING. It is not easy to extract data from for subject groups in the following year.

31. They can be sorted by all criteria live on OneSchool • They can be downloaded as Excel spreadsheets to allow “off-line” manipulation, or as PDF files. • They have multiple pages Non-graphical reports are interactive within OneSchool.

32. If you export a report after sorting, it retains the sorted order in the exported document. • If you export the individual students item results as an excel spreadsheet it contains one worksheet per student.

33. In live views of OneSchool reports that show item numbers, pointing at the item gives information about the item.

34. In live views of OneSchool reports that show item numbers, clicking on the item gives discussion and teaching ideas.

35. How is Browns Plains High addressing the need to engage staff in data analysis.? A “DataWise” team has ben established to analyse whole school data and to develop strategies for staff engagement at a productive but sustainable level.

36. We are presently formulating a whole school approach using selected NAPLAN data. The example to follow has not been finalised for use, and is subject to further development

37. We are currently developing a document template to allow staff to quickly and easily • access NAPLAN data about the classes they are presently teaching • Identify both the general ability of the class group and those students needing extension or support within the class group • Identify differentiation strategies for those students.

39. Live links to take staff directly to the appropriate reports

40. Sorting the results by “difference from Nation” allows teachers to identify areas of poorer performance, using “item information” facility

41. Visual scan of results will allow identification of students in need of extension or support

43. For our first trial of this process, we intend to have teachers examine only NAPLAN data. In Semester 2, students will continue with the same teacher as in Semester 1, who will have prior knowledge of assessment outcomes. Future iterations of this process will include review of assessment data from previous years, and other test data such as Torch tests for literacy and standardised Numeracy testing.

44. If you wish to view my presentations, they can be downloaded from johnbright.yolasite.com