1 / 39

Introduction to CEM Secondary Information Systems

Introduction to CEM Secondary Information Systems. Dr Robert Clark ALIS Project Manager. The Analysis. Linear Least Squares Regression. Subject X. -ve VA. +ve VA. 0 2 4 6 8. Residuals. Regression Line (…Trend Line, Line of Best Fit) Outcome = gradient x baseline + intercept

mannix-duke
Download Presentation

Introduction to CEM Secondary Information Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to CEM Secondary Information Systems Dr Robert Clark ALIS Project Manager

  2. The Analysis

  3. . Linear Least Squares Regression Subject X -ve VA +ve VA 0 2 4 6 8 Residuals Regression Line (…Trend Line, Line of Best Fit) Outcome = gradient x baseline + intercept Correlation Coefficient (~ 0.7)

  4. A* B C Alf Bob Subject A D Result Chris E Subject B F G U Low Ability Average Ability High Ability Baseline Score Measuring Value-Added – An Example National Trend ‘Average’ Student -ve (- 2 grades) +ve (+ 2 grades) The position of the national trend line is of critical importance

  5. Photography Sociology English Lit Psychology Maths Physics Latin Some Subjects are More Equal than Others…. A-Level (Alis) A* A B Grade > 1 grade C D E C B A A* Average GCSE

  6. Biology Business and Management Chemistry Design Technology Economics English_A1 Film French_B Geography History Mathematics Music Philosophy Physics Psychology Spanish_B Theatre Arts Visual Arts Some Subjects are More Equal than Others…. IB (Alis) 7 6 Grade 5 4 C B A A* Average (I)GCSE Score

  7. A* A B Art & Design Biology GCSE Grades Chemistry C Economics English French Geography German D History Ict Mathematics Media Studies Music E Physical Education Physics Religious Studies Science (Double) Spanish F Test Score Some Subjects are More Equal than Others…. GCSE (MidYIS or Yellis) 1 grade

  8. Standardisation of Residuals • (Raw) Residuals can be used to examine an individual’s performance • Standardised Residuals are used to compare performance of groups • Standardised Residuals are independent of year or qualification type • For a class, subject, department or whole institution the Average Standardised Residual is the ‘Value-Added Score’ • Standardised Residual = Residual / Standard Deviation (National Sample) • When using Standardised Residuals then for an individual subject where N = number of results in the group (for combinations of subjects consult the relevant project) • 95% Confidence Limit = 2.0 x Standard Error • 99% Confidence Limit = 2.6 x Standard Error • 99.7% Confidence Limit = 3.0 x Standard Error

  9. The Projects

  10. Alis : yr 12/13 Yellis : yr 10/11 Secondary Age Range Projects Insight : Yr 9 MidYIS : Yr 7/8/9 InCAS : Primary Pips : Primary

  11. Measure Baseline Prediction Reports Collect Results Value-Added Feedback Typical Timeline Autumn Term August September

  12. Baseline Measurement

  13. A / AS / IB etc GCSE Alis Paper test (TDA) or Computer Adaptive Baseline Test GCSE Yellis Paper test or Computer Adaptive Baseline Test MidYIS Paper test or Computer Adaptive Baseline Test Baselines Year 13 Year 12 Year 11 Year 10 Combines curriculum tests with developed ability (end Y9 & end Y8) INSIGHT Year 9 Year 8 (+ additional) Year 7

  14. Problems with Key Stage Baselines • Not all students have KS baselines • no KS2 / KS3 • Foreign Students • (Vocational Students) • (Adult Learners) • KS exams do not always represent ‘Start of Course’ ability • Post-16 : Year(s) out or intermediate years • Prior Value-Added • Can you add value at every Key Stage ? • Under achievement leading to under expectation • One teacher’s output = another teacher’s input • A level playing field ? • Teaching to the test • Does performance represent ability, effort or exam technique ? • Aptitude & fluency vs Achievement & knowledge

  15. Average GCSE = 6 Average GCSE = 6 Average GCSE = 6 The Effect of Prior Value Added Beyond Expectation +ve Value-Added In line with Expectation 0 Value-Added Below Expectation -ve Value-Added Do these 3 students all have the same ability ?

  16. yrs 7 / 8 / S1 / 9 / S2 yrs 10 / 11 yrs 12 / 13 + • Although Key Stage baselines can be a very good indicator of potential attainment, by themselves they are not sufficient. • Key Stage baselines are confounded by the effects of prior treatment. Need for independent, non-curriculum embedded baseline tests Online Computer Adaptive or Traditional Paper

  17. The Computer Adaptive Test • Test performed online – results automatically transmitted to CEM. • LAN version available if Web access unreliable. • Minimal installation / setup required - if any. • Adaptive – difficulty of questions changes in relation to ability of student. • Efficient – no time wasted answering questions that are far too easy or difficult. • Wider range of ability • Less stressful on students – more enjoyable experience than paper test. • Less demanding invigilation. • Cheaper ! In 2009 / 2010 over 200,000 students across yrs 7-13 sat this test Try it yourself at www.inturproject.org/demos

  18. Baseline Feedback Reports, Graphs & Predictions

  19. IPRs (Individual Pupil Record Sheets) Look for sections that are inconsistent Also available based on MidYIS, Alis & INSIGHT scores

  20. Intake Profiles Also available based on MidYIS, Yellis and INSIGHT scores

  21. Intake Profiles (Historical)

  22. Intake Profiles – 7 Band for Independent Schools (Midyis)

  23. Average performance by similar students in past exams Predictions • Predictions are available in the following forms: • Formal Reports (Alis) • Spreadsheets (All Projects) • Paris Software (Alis / Yellis / MidYIS) • Feedback is available on Web / CD

  24. Predictions – MidYIS example Similar spreadsheets available from Yellis and INSIGHT

  25. Predictions - Alis example

  26. Value-Added Feedback Reports & Graphs

  27. Value Added Feedback… What is my score ? does it matter? Statistical Process Control (SPC) Chart 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Year

  28. Subject Summary Standardised Residual Graph

  29. Grade Points Equivalent Baseline Score The Scatter Plot Look for Patterns… General Underachievement / over achievement ? Do any groups of students stand out ? – high ability vs low ability ? – male vs female ?

  30. Other things to look for… Why did these students do so badly ? Why did this student do so well ? How did they do in their other subjects ?

  31. Independent Schools Analysis

  32. MidYIS provides… • Independent schools baseline standardisation • Independent schools specific predictions & chances graphs • 7 Band Intake profiles • Independent schools specific value-added analysis Alis provides… • Independent schools specific value-added analysis Q. Which one should I use – Independent only analysis or all schools analysis ? A. It depends on what question you are asking…

  33. PARIS Software

  34. PARIS is ….. • Software to install and use in school • An interactive reporting tool • Included free with Alis / Yellis / MidYIS PARIS analyses ….. • Potential Performance • Intermediate Performance • Actual Performance PARIS provides ….. • Student level reports • Subject Level reports • Institution Level Reports

  35. Attitudes

  36. There is more to school / college than exams…. • Student attitudes • Student Welfare & Safety • Non-academic activities • Support • Social and personal development • Parental Survey • Induction Survey Self Evaluation (Every Child Matters) Attitudinal MidYIS INSIGHT Attitudinal Yellis Full ALIS Try it yourself at www.inturproject.org/demos

  37. Other Issues

  38. Points to mull over… • Independence – no agendas • Transparency of Analysis • Self Evaluation • Straightforward and standardised administration • Prompt Feedback • Full working hours phone / email support • Student focus • Replacement for KS3 • Innovative online adaptive testing available – student experience • Longitudinal analysis with appropriate error backgrounds • Non-curriculum embedded baselines available • Attitudinal surveys available – Every Child Matters…

  39. Dr Robert Clark Alis Project Manager robert.clark@cem.dur.ac.uk 0191 33 44 193

More Related