Universiteit Maastricht

1. Universiteit Maastricht Progress Testing Update from collaborating schools in the Netherlands Lambert Schuwirth on behaff of the taskforce interuniveristy progress test

2. PT collaboration in the Netherlands Universiteit Maastricht overview - history - current situation - future

3. PT collaboration in the Netherlands Universiteit Maastricht overview - history - current situation - future

4. PT collaboration in the Netherlands Universiteit Maastricht

5. PT collaboration in the Netherlands Universiteit Maastricht - PT started in 1977 - 250 true-false questions per test - extensive item production - extensive quality control

6. anatomy physiology int medicine item analyses student comments surgery psychology review committee test administration item pool item bank PT collaboration in the Netherlands Universiteit Maastricht

7. PT collaboration in the Netherlands Universiteit Maastricht costs item production: 1250 x 1 hour = 0.76 fte item review: 6 x .3 fte + .5 fte = 2.3 fte admin support: = 0.8 fte total: 3.06 SS + .8 AS = 3.86 fte

8. PT collaboration in the Netherlands Universiteit Maastricht overview - history - current situation - future

9. PT collaboration in the Netherlands Universiteit Maastricht

10. PT collaboration in the Netherlands Universiteit Maastricht - 5 participating schools - Groningen, Nijmegen, Leiden, Amsterdam - collaborative test production - collaborative quality control - collaborative research endeavours

11. anatomy physiology int medicine item analyses student comments surgery psychology review committee test administration item pool item bank PT collaboration in the Netherlands Universiteit Maastricht at each school

12. telephone conference review committee review committee review committee review committee final test content PT collaboration in the Netherlands Universiteit Maastricht

13. PT collaboration in the Netherlands Universiteit Maastricht - roughly 10,000 candidates per test - four tests per year - each year 24 comparisons between schools - individual comparisons fluctuate - clear trends visible

14. PT collaboration in the Netherlands Universiteit Maastricht ☺ 40 U1 30 U2 U3  20 10 0 -10 0 4 8 12 16 20 24 test number adapted from: Muijtjens AMM, Schuwirth LWT, Cohen-Schotanus J, Thoben AJNM, & Van der Vleuten CPM. (2008). Benchmarking by cross-institutional comparison of student achievement in a progress test. Medical Education, 42(1), 82-88.

15. 1 2 3 4 5 6 1 2 3 4 5 6 04-05 03-04 PT collaboration in the Netherlands Universiteit Maastricht Cumulated deviation score U1 1 U2 U3 0 -1 Year group 1 2 3 4 5 6 1 2 3 4 5 6 02-03 Acad. year 01-02 adapted from: Muijtjens AMM, Schuwirth LWT, Cohen-Schotanus J, Thoben AJNM, & Van der Vleuten CPM. (2008). Benchmarking by cross-institutional comparison of student achievement in a progress test. Medical Education, 42(1), 82-88.

16. PT collaboration in the Netherlands Universiteit Maastricht disadvantages differences in quality, lower control on quality logistical complications (venues, dates, mail, confidentiality, vulnerability, etc.) differences in regulations pertaining to PT difficult to handle in local regulations unclarity about year groups of students ‘home’ effects

17. PT collaboration in the Netherlands Universiteit Maastricht advantages good comparative material reduction of costs: 1.15 review, 0.225 as, 0.28 production = 1.655 stable relative norm wider acceptance room for further developments ample data for multicentre high power research

18. PT collaboration in the Netherlands Universiteit Maastricht overview history current situation future

19. PT collaboration in the Netherlands Universiteit Maastricht future developments query-based feedback test equating  IRT modeling Angoff procedures sequential or adaptive testing relevance evaluation involvement of more partners?

20. PT collaboration in the Netherlands Universiteit Maastricht THANK YOU