Course Review on Assessment

1. Course Review on Assessment Professor Brenda Smith

2. THE TESTA MODEL – FOR ENHANCEMENT • It is intended to map assessment patterns across a programme • It looks back at modules over a 3 year degree course to chart a typical students experience 10 steps to auditing assessment across a programme

3. TESTA: Transforming the experience of students through assessment • Funded by the National Teaching Fellowship Scheme • Seeks to develop course assessment interventions and to evaluate them • Involves 4 universities and led by Graham Gibbs

4. TESTA Model It maps: • Volume of summative & formative assessment • Variety of assessment methods • Percentage from coursework and exams • Volume of written and oral feedback • How criteria contribute to student clarity about goals and standards

5. Caveats… • Gives an overview from a course leader or course team perspective • There is often a gap between documentation & the reality of the student experience • It works best when triangulated with other data like focus groups with students and the Assessment Experience Questionnaire (AEQ)

6. TEN STEPS - STEP 1 Select a named programme What is the typical student cohort • Number of students in each year • What is their previous educational experience? • What are students career aspirations? Year 1 Year 2 Year 3

7. STEP 2 • How many modules are there in the first year of the programme? • What is their credit weighting? 20 Credits 40 Credits

8. Business Studies Year 1 •  Management Accounting•  Contract Law•  Foundations of Marketing•  Economics for Business•  Managing People•  Quantitative Business Methods

9. STEP 3 Describe the nature of assessment in each module • Type of assessment • Is it formative and summative? Now tally the summative, formative and variety of assessment methods

10. STEP 4 • Repeat for the second and third years • Tally the number of summative, formative and the variety • Tally the number of assessments by exam, and work out the relative percentage of course work to exams

11. STEP 5 • For each assessed task estimate how long it takes from the submission of an assessed task to the return of feedback and marks • Are feedback and marks separated? • What is the system for the collection of assessed work? • Is certain feedback and the mark returned faster than others? E.g presentations

12. STEP 5 continued…. • Are feedback and marks released as provisional? • Is formative feedback returned more quickly than summative feedback? • Are there different variables e.g. 80% of written work is returned after 21 days, & 20% is presentation, returned within 1 day? • WORK OUT THE OVERALL RETURN FACTOR

13. STEP 6 Work out as best you can how much oral feedback a student is likely to receive each year and over the degree • Tutorial provision before & after assessment points, plus estimated take up of these % wise & length of tutorial • You may need to explore how assessment is handed back • May involve looking at numbers of students called back after a failed submission • May involve quantifying proportion of oral feedback given through digital technology- (MP3 files & smart phones) & proportion who listen, oral feedback given in whole class sessions and on placements etc

14. STEP 7 Look at explicit assessment criteria and learning outcomes in the programme documentation • How do written statements appear in mapping from programme to module to task? • Is there alignment between tasks & intended outcomes? • How do students know what ‘good’ is in relation to assessment tasks

15. STEP 7 continued… • Does the programme have a system wide approach to developing the student ‘nose’ for what constitutes a good essay, presentation, poster etc? • How do feedback processes clarify goals and standards?

16. STEP 8 • Using the sequence guide (given as a separate sheet) map the sequence of different assessment tasks across modules and levels. • Describe how assessment articulates across the programme , feeding forward into the next assessment, providing practice and allowing students to develop their performance

17. STEP 9 Gather together 15-20 cover sheets and if possible scripts with comments on them from each year – 1st, 2nd & 3rd. • Count the number of words on the cover sheets and scripts • Add 1st, 2nd & 3rd year total words • Divide the total by the number of student cover sheets • Multiply the average number of words of written feedback by the total volume of summative feedback • This is the total volume of written work a student might expect over the course of a 3 year programme

18. STEP 10 • Put all the data together in a document • Now working with the course team what are you really pleased about? • What might you want to do differently? • How might you run some student focus groups to triangulate this information? Ref: De Tansy Jessop, TESTA

19. Table 2 Range of characteristics of assessment environments found in different degree programmes Characteristic of assessment environment Minimum Maximum Percentage of degree marks derived from examinations 17% 100% Percentage of degree marks derived from coursework 0% 83% Total number of times work marked per student 11 95 Variety of assessment methods 1 18 Total number of formative-only assessments per student 2 134 Total number of words of written feedback per student 2,700 10,350 Total number of hours of oral feedback per student 3 68 Average number of days between submission of assignment and feedback 1 28 REF: Graham Gibbs “Using assessment to support student learning” Leeds Metropolitan University

20. Positive student learning responses The following programme level assessment characteristics have been found to be associated with a variety of positive student learning responses: • a high volume of formative-only assessment • a high volume of oral feedback • timely feedback … and to a lesser extent: • a high volume of written feedback • a low proportion of marks from coursework

21. Negative student learning responses The following programme level assessment characteristics have been found to be associated with a variety of negative student learning responses: • a high volume of summative assessment • a wide variety of types of assessment • highly explicit criteria and standards • highly ‘aligned’ assessment (with different assessments and criteria associated with each learning outcome)

22. Figure 1: A student’s estimate of his weekly study effort (in hours) on a programme with no assignments and an exam in week 12. Total study effort = 68 hours

23. Figure 2: A student’s estimate of her weekly study effort (in hours) on a programme with three assignments due in weeks 4, 7 and 10 and an exam in week 12. Total study effort = 78 hours Ref: Using assessment to support student learning (2010)By Graham Gibbs ISBN 978-1-907240-06-5

24. TESTA Programme PortraitWombledown University, Creative Arts Degree (CAD)

25. TESTA Programme PortraitWombledown University, Creative Arts Degree (CAD) • On average, a CAD student received 214 words of feedback on each summative assessment, low oral feedback from tutors, and no formative assessment was formalised in the documentation. The programme was high in variety, and it was the quickest on return of assessed work

26. Focus Group Data USE OF FEEDBACK • Students use feedback to feed-forward • For some module feedback does not feed forward • Peer feedback is highly valued & integral to the program • It takes time and practice to give good feedback • Low stakes, informal feedback is valued • Comments without marks increase use of feedback • Marks without feedback diminish use of feedback