MENO An admission test experiment at the Vrije Universiteit Brussel

1. MENOAn admission test experiment at the Vrije Universiteit Brussel Raoul Van Esbroeck

2. Why an admission test in Flanders? • Open admission • High failure rate • Costs for society • Costs for the individual student • Need for more efficiency • Admission test a possible option Vrije Universiteit Brussel

3. Why the MENO-test? • UCLES (University of Cambridge Local Examination Syndicate) • Skills crucial to success in HE • Limited to 50 multiple choice items • Critical Thinking (CT) with 3 subscales • Problem Solving (PS) with 3 subscales • Good reliability and validity • Limited bias (language, age, ethnic origin) • Gender and SES bias? Vrije Universiteit Brussel

4. Admission test experiment • OBPWO project (Promoter: Prof. M. De Metsenaere) • Questionnaire • Social and cultural variables • MENO- test • Administered in class during first week • Cohort of entering Freshmen in 95-96 • Respondents: 85% of the target group (N = 1000) Vrije Universiteit Brussel

5. Does MENO allow correct classification? • Errors in classification • False positives (FP) • False negatives (FN) • Total number of errors % refused %FN %FP 10 6 87 25 18 63 50 39 39 • Conclusion: too many errors Vrije Universiteit Brussel

6. Has the test an added value? • Can the test predict academic success better than SE results? • A model with 4 variables: • SE option, class rank, graduating on time • MENO • Explains 18.7% of variance • MENO represents only 1.75% in this model • Conclusion: only limited added value Vrije Universiteit Brussel

7. Bias? • There is a gender bias • Girls would be more refused than boys • FN error is higher with girls • FP error is higher with boys • There is a SES bias • Lower SES will be more refused than higher SES groups • FN error is higher with lower SES groups • FP error is higher with higher SES groups Vrije Universiteit Brussel

8. Increase of success rate? • There is limited increase • If 25% of the lowest MENO-scores are eliminated success rate increases from 49% to 58% • What about errors? Vrije Universiteit Brussel

9. Conclusion on MENO • MENO is not an acceptable admission test • Too many errors in classification • Limited added value • There is gender and SES bias • The gains are limited • Using this test would be unethical Vrije Universiteit Brussel

10. Are other tests better? • There are indications that the MENO in the UK is also biased • The Chrysostomos test (P. Janssen, KULeuven) also faces classification errors • A “Pass” (50%) score on the test leads to 37.6% of classification errors • A 60% score on the test leads to 30.9% of classification errors (with about 5% of FN) • Similar problems exist in other countries where admission tests are used Vrije Universiteit Brussel

11. General conclusion • There is no need for admission tests because • There is limited added value • Too many errors are made • There is bias • The gains in success rate are limited • The costs for the tests are extremely high • There is no real societal benefit if FN are to be avoided • There will be too many wrong decisions if you want to have real gains Vrije Universiteit Brussel

12. Personal reflection • The use of general admission tests for selection is unacceptable and unethical • Such tests may be excellent for guidance purposes Vrije Universiteit Brussel