100 likes | 226 Views
A simulation of a Flexilevel test. Andrew Pyper, Mariana Lilley, Paul Wernick and Amanda Jefferies. What is a Flexilevel Test?. Background. We have carried out studies at different levels and in different educational contexts Levels 4 and 7 Pilot study Formative assessment
E N D
A simulation of a Flexilevel test Andrew Pyper, Mariana Lilley, Paul Wernick and Amanda Jefferies
Background • We have carried out studies at different levels and in different educational contexts • Levels 4 and 7 • Pilot study • Formative assessment • Summative assessment
Attitude • We asked students and tutors what they thought about the Flexilevel approach… • Focus groups • Surveys • Overall • Formative- sounds good • Summative- not so keen
Why simulation? • We have quite a lot of data from studies in real educational contexts, so why this study? • Practical considerations associated with: • Setting up an assessment • Setting up the lab • Timing- limited number of opportunities for running assessments and we need data • Opportunity to see what the results look like without examinees actually experiencing the flexilevel test
Why simulation? • We need more data – lots of points of interest. • Requirements for calibration • How many items are needed to obtain the same results as standard CBT • The influence of different contexts
Preparing the simulation • Examinee responses from two different computer-based tests (CBTs) were collated for analysis. • Level 4 Databases • Level 6 Databases • Calculate the difficulty of each item
Applying the flexilevel algorithm • The items were ranked according to their difficulty • The flexilevel algorithm was applied to the selection of the items • Stop the simulation after half (or almost half) the number of items are presented
Results • In the case of Databases (Level 4), there was a strong, positive correlation between the CBT and Flexilevel scores, which was statistically significant (rs(9) = 0.949, p < 0.01). • In the case of Databases (Level 6), there was also a strong, positive correlation between the CBT and Flexilevel scores, which was statistically significant (rs(63) = 0.930, p < 0.01).
Next Work • Further simulation studies • Further studies in real educational contexts- points of interest include: • Formative assessment in mobile contexts • Applying the flexilevel approach to learning activities more generally than assessment