Developing assessment in Engineering and Physical Sciences

1. Developing assessment in Engineering and Physical Sciences a really practical workshop Sue Bloxham, s.bloxham@cumbria.ac.uk

2. Workshop programme Three overlapping sections: • developing authentic and synoptic assessment; • embedding formative assessment in teaching and learning activities; • helping students understand assessment requirements. And lots of chance to consider these ideas in relation to your programmes and modules

3. Authentic and Synoptic learning • Promotes productive, worthwhile approaches to learning (unlike exams), encourages higher order learning (Sambell et al, 2013) • It creates assessment which is inherently meaningful, interesting, relevant and have long term value (Sambell et al, 2013) • Can focus on programme learning outcomes and assist students to integrate learning from different modules at each stage of their course

4. Existing examples at Queenshttp://www.queens-racing.com • Developing Key Skills through an Authentic Design-Build Project – Formula style racing car • Project 3 assessment: individual report, peer assessment, oral presentation and oral examination; • Project 4 assessment: group report, peer assessment oral presentation. Professional Studies 4: group report and presentation. • Whilst this is a fairly ambitious authentic assessment and not scalable for most modules, there are plenty of simpler ways of creating more authentic assessment. Dr Geoffrey Cunningham, School of Mechanical and Manufacturing Engineering, Queen’s

5. Authentic assessment methods • Writing tasks: press releases, executive summaries, technical reports, information sheets, Wikipedia entry (authentic communication). • Video about a specific topic – Youtube? (developing new skills, communication, demonstrating new understanding) • Research Grant applications (lots of learning, less marking) • Lay commentary on specialist material, e.g. News & Views articles,(being able to explain things to non specialists – demonstrate understanding) • Poster or E-poster (synthesising material, presenting information clearly & concisely) • Presentation – ‘explanation’ as learning • Solve real world problems (analysis, application of knowledge,problem solving)

6. Biodiversity & evolution: news & views articles (Lesley Morrell – Hull) • Focus on how scientists research scientific questions based on 8 research seminars • Assessment: • 5% participation in seminar, • 25% for a poster (defended) based on a recent scientific paper (prep for final year assessment of own projects), • 70% for 8 ‘News and Views’ type journal articles – ie 300-500 word summaries of interesting & important new papers • Articles based on one of the papers presented in each seminar. Morrell, 2014 Morrell, 2014

7. Morrell – News & Views feedback “This has been the most valuable module I have taken in my degree. It has improved the way I read papers by changing the way I find the most important information in articles. I have therefore been able to apply this to other modules. The amount of feedback has been excellent and the class participation has improved the way I listen in lectures”.

8. Learning from real cases: built environment • Health and safety management module • Assessment: research, prepare and deliver a presentation on a real-life accident that students find and research (last 12 months prevents plagiarism!) • Presentation (could be a report) to include relevant regulations, codes and guidance; good management practice for the hazard; and create a hazard control risk assessment. • Relevant for work; sharing with other students provided useful learning about other hazards and their safe management. (Sambell et al 2013; 23)

9. Enhancing the learning value from final year projects: the mini conference • Students complete a ‘capstone’ project which integrates and builds on learning across the course; • Students submit shorter reports than a traditional dissertation; • Instead, they attend and present (papers or posters) at a mini-conference attended by staff and students, even local employers • Presenting encouraged sounder learning as students had to be able to explain and answer questions about their work (Sambell et al 2013, 26) • Mirrored real research

10. And similar pattern across all years with increasing specialisation • Challenges: year 1 students work in interdisciplinary groups. - assessed by video in relation to: • Best Sustainable Energy Design Solution • Best Technical Argument with Social, Environmental and Economic Consideration • Most Engaging Video examples of student videos

11. Programmatic assessment:Graduate entry program Maastricht - medicine • Stopped thinking in terms of individual assessment methods to test student achievement in specific domains. • Replaced by a systematic and programmatic approach, longitudinally oriented recognising that passing a test once does not guarantee competence. • Many methods of assessment are used • old and new; standardized and unstandardized, experts, peers, self, tests, OCSEs, in practice, etc) • Focus is on programme outcomes in terms of achieving competencies • Assessment structured by an e-portfolio where all assessment data is held – information rich system • Professional judgment is imperative (similar to any professional practice). • Subjectivity is dealt with through sampling and procedural bias reduction methods (not with standardization or objectification) • Decision-making by committee based on portfolio information • Assessment becomes personalised – • remediation of learning may be prescribed Van der Veuten, 2012

12. Discussion Talking with subject colleagues • What aspects of our existing assessment methods are authentic and synoptic; • Should we be trying to make more assessment authentic and synoptic; • Can we draw on any of the examples in the presentation?

13. Assessment as (active) learning in lectures https://www.youtube.com/watch?v=TkQ_v2PB00Q&feature=youtu.be Note the opportunity for students to: • develop learning through explaining and listening; • Get immediate feedback on whether they understand something And for the lecturer to: • Get immediate feedback on whether the students have understood • Create engaged

14. Peer assessment in lab reports Peer assessment of lab reports in pharmacy • Weekly reports – big marking load, little improvement over term. • 1st week, students marked exemplars of mixed quality followed by teacher explanation • Thereafter, 1st 15 mins each session, peer marking of reports and immediate return. Tutor moderated a sample – students accurate or tougher! • 20% increase in marks • More effective for learning than tutor feedback Gibbs (1999)

15. News & views articles – embedded formative feedback • Students submit 8 x 500 word News & Views articles; • All get detailed feedback on the first in time for the second. • After that, only a small proportion of assignments are marked but all these, anonymised and with feedback, are posted on the VLE so students can learn from the success and mistakes of others • Student select two (which they can rework) for summative mark – those who decided their best did better than those playing safe with tutor marked work. • Social pressure to do good work – it may turn up on the VLE. • Students marks improved through the term, they appreciated the extensive feedback. Morrell, 2014

16. 3rd year Medicine: developing research proposals • Task is to create 1200 word research proposal in an area of medicine chosen by the student; • Supportive lecture series on doing medical research available; • Students submit first draft, part way through, to VLE – sent anonymously to two students for comment; • Summative assessment is final proposal plus brief record of response to peer review. • Final ‘package’ to be checked also includes draft proposal, both peer reviews, the student’s own reviews of other work

17. Tutor friendly formative feedback on written work • Tutor posts good examples of completed tasks on VLE; • Students peer assess tasks using assessment criteria, either in or out of the classroom; • Tutors give feedback on work in class; • Tasks are done on-line (MCQs), auto marked giving immediate feedback – maybe useful for technical knowledge • Tutors put main effort into marking drafts (agreed with examiner), just checking for change and putting mark on final piece

18. Engaging students in formative assessment – key requirements • It clearly feeds into summative assessment tasks; • The students must submit it in some way (bring to class, post on line, hand it in, do in class) and action is taken if they don’t; • Students receive useful feedback on it; • It is not contaminated by summative purposes.

19. Selling peer assessment • Evidence shows students find their peers a useful and more approachable source of help with assignments but we need to stress the main value in peer assessment is standing in the shoes of the assessor – not being assessed – because: • learning about standards – absolutely crucial to making progress and understanding feedback • Seeing other ways of going about the task – develops strategies for taking action • Key employability skill – being able to judge own performance and assessing and giving feedback to others • More opportunity for dialogue • Chance for more formative feedback Peer assessment needs to become a regular feature of programmes so that it is taken seriously and taken for granted as part of learning at this level.

20. Discussion Talking with subject colleagues • What opportunities do our students receive for formative feedback? • Can we draw on any of the examples in the presentation to embed formative feedback in our teaching and learning? • How do we monitor whether students take any notice of feedback?

21. What we say and what we do What our guidance implies Reality of academic judgement We can make standards explicit Markers’ standards are internalised and tacit (difficult to express) Norm-referencing of judgement needed/ interpretation of criteria Criterion-referenced grading Analytical judgement Holistic professional judgement Marker variation There is a ‘right’ mark We struggle to communicate the reality of our professional judgement in a way which is understandable and credible to students? Bloxham (2012)

22. Example 1: Scientific reports • Biological sciences • Students mark two exemplars of a scientific report – one high & one medium standard • Student discussion about marks given and justification • Tutor tells and explains actual marks • Followed by students peer reviewing each others’ draft assignments • Extremely positively evaluated by students – increased confidence & reduction in anxiety Yucel et al (2014)

23. Example 3: Bioscience exams • Biological sciences- 12 modules • UG & PG students provided with exemplar answers on VLE – average, good and excellent standards • Exemplars annotated with markers comments • Students accessing exemplars had improved marks • Students and tutors positive Scoles, Huxham, McArthur 2013 • Strongest effect found in large classes • Particularly useful for students studying other subjects

24. General benefits of exemplars • Students & staff very positive • Enable students to better grasp overall quality, structure, language, style than lists of individual criteria; • Contributes to confidence to tackle assessments; • Evidence of improved marks across different ability levels; • Encourages holistic judgement; • Helps students recognise different ‘expressions of quality’.

25. Importance of active engagement • Marking exercises and structured discussion contribute to helping students learn from exemplars; • Force students to think about standards - helps build their tacit knowledge; • Expert explanation/ annotation following marking exercise helps to make criteria ‘concrete’ – makes standards visible; • Marking exemplars removes ‘emotionality’ of peer review.

26. Copying? • Exemplars should not be model answers which invite ‘mimicry’; • Exercises should encourage questioning and critique of exemplars; • Experiments avoided providing outstanding exemplars; • Studies show no evidence of copying exemplars.

27. Creating exemplars • Use work similar to that produced by students at that level – range of standards; • Multiple examples of same standard to recognise multiple ways of achieving quality; • Short examples which focus on ‘archetypal problems’ or ‘good practices’ may be more useful (Handley & Williams 2011)

28. Task Working in groups • Select an aspect of assessment in your subject that you think could benefit from some development. • Come up with some proposals. • Particularly think about assessment tasks which: • Are synoptic: contributing to programme level outcomes; • Are authentic; • Embed formative assessment; • Help students understand the criteria. Be prepared to present your assessment plan to the group

29. Assignment Brief Communication http://assignmentbriefdesign.weebly.com

30. References Bloxham (2012) ‘You can see the quality in front of your eyes’: grounding academic standards between rationality and interpretation Quality in Higher Education, 18 (2) 185-204 Gibbs, G. (1999) Using Assessment Strategically to change the way students learn, in Brown, S. and Glasner, A. Assessment Matters in Higher Education: Choosing and Using Diverse Approaches. Buckingham: SRHE / Open University Press:41-53. Morrell LJ (2014) Use of feed­forward mechanisms in a novel research‐led module. Bioscience Education 22: 70-81. Handley, K & Williams, L (2011) From copying to learning: using exemplars to engage students with assessment criteria and feedback. Assessment & Evaluation in Higher Education 36 (1) 95- 108 Sambell, K., McDowell, E and Montgomery, C (2013) Assessment for Learning in Higher Education London: Routledge Scoles, J et al (2013) No longer exempt from good practice: using exemplars to close the feedback gap for exams. Assessment & evaluation in higher education, 38 (6) 631-645 Van der Vleuten, C.P.M. et al (2012) A model for programmatic assessment fit for purpose. Medical Teacher 34(3) Yucel, R et al (2014) The road to self assessment :exemplar marking before peer review develops first-year students capacity to judge the quality of a scientific report, Assessment & evaluation in higher education, 39 (8) 971-986