Quality Assessment The Blueprinting Approach

1. Quality Assessment The Blueprinting Approach Assoc Prof Craig Zimitat Director, Medical Education Unit

2. Overview Background What is a blueprint? Why use a blueprint? What blueprints do we need? Examples

3. Perspectives • Teachers and students tend to approach their tasks from different perspectives. • Teachers focus on outcomes and content • Students focus on assessment • A mismatch between outcomes and assessment causes confusion for everyone in the teaching and learning enterprise.

4. Teaching & Learning Activities Intended Learning Outcomes Assessment Tasks Curriculum Alignment • The curriculum is designed so that the learning activities and assessment tasks are clearly related to the intended learning outcomes.

5. Teaching & Learning Activities Intended Learning Outcomes Assessment Tasks Curriculum Alignment • Too often assessment tasks are not related to stated outcomes, not valid for the outcome, or incongruent with teaching methods.

6. What is a blueprint ? A Blueprint is a plan that explicitly relates outcomes and assessment strategies . guides the development of assessment systems at a course level guides development of assessment as unit level guides the appropriate use of individual instruments It provides evidence of the design making processes involved in choosing appropriate assessment tasks, sampling of subject matter and balance.

7. Why use a blueprint? To explicitly align assessment processes with desired learning outcomes in terms of: What will be assessed How it will be assessed To ensure an appropriate range of objectives, content and weightings are addressed when devising specific examination instruments Allows sharing and communication with others Provides a basis for consistency over time

8. Managing assessment Identifies gaps in current assessment Develops shared understanding amongst assessors – in and across year groups Improves content validity Provides a framework for feedback to students Links with IDEAL database of questions Part of a QA Process

9. What will be assessed? State outcomes and define in terms of competencies (Curriculum mapping) Identify appropriate assessment strategies Look at timing Include other issues e.g. setting, system, patient age profile etc.

10. How it will be assessed Select the instruments that best assess the competencies defined Select instruments that are consistent with teaching strategies and course philosophy Blueprint can include: Formative assessment tasks Summative assessment tasks Details about timing

11. Managing assessment Identifies gaps in current assessment Develops shared understanding Improves content validity Links with IDEAL database of questions Provides a framework for discussion and feedback QA Process

12. What blueprints do we need? At least three Level 1: Overall course (MGP) Level 2: Unit level consider year level for Y3 rotations Level 3: Instrument level Often OSCE will have a separate blueprint

13. Curriculum map

14. Blueprint for an assessment system

15. Blueprint – Y3 Rotations OSCE

16. Blueprint – Rotation OSCE x Presentation

17. QA of Current Assessment Map distribution of questions by instrument Map distribution of questions by topic Compare this distribution with curriculum map Look for links: then appropriateness, balance and gaps What evidence guides current practice? BEECH data Medical Board and MDA data Tasmanian vs Australian data