Assessing student learning in diverse ways: Portfolios

1. Assessing student learning in diverse ways: Portfolios Rosalind Duhs Centre for the Advancement of Learning and Teaching (CALT) This document is licensed under the Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales license, available at http://creativecommons.org/licenses/by-nc-sa/2.0/uk/.

2. Introductions • Name • Role • Discipline • Interest in portfolios for assessment • Any concerns?

3. Session intended learning outcomesBy the end of the session, participants will be able to: • Plan portfolio assessment in line with intended learning outcomes and course learning and teaching activities to enhance higher order learning • Evaluate the potential impact of portfolios on student learning • Outline a plan of action: how could your students work with portfolios?

4. Portfolios: an introduction

5. What are portfolios? A portfolio can be viewed as a collection of papers and other forms of evidence that learning has taken place. From: http://www.medev.ac.uk/resources/resources/features/AMEE_summaries

6. What might portfolios contain? (Baume 2001) A selection of work to evidence the attainment of learning outcomes (word-based, images, films, sound, webpages) Engineering – analyses and designs Social sciences – reports and essays Scientists – lab reports Reflection on group work, work placement, clinical practice, projects Synoptic portfolios – a review of achievement and learning throughout a student’s programme of study

7. Portfolios: the essential ingredient? An analysis and evaluation of the content in relation to the intended learning outcomes of a programme, course or module - Students explain how the portfolio provides evidence of their learning, referring to the contents

8. PORTFOLIOS: Summative and Formative assessment • Summative assessment counts towards final results in relation to learning outcomes • Formative assessment does not count towards final course grades, but measures progress and provides students with valuable feedback • Summative assessment should also be formative

9. Portfolios can build on a dialogic feedback system embedding drafting and redrafting Rosalind Duhs 2010

10. What can portfolios do? (Baume 2001)Summary • support the development, demonstration and valid assessment of a wide range of personal, professional and academic capabilities, both inside and outside a programme of study; (Personal Development Planning PDP) • provide evidence of work done and learning achieved; • show reflection on and analysis of evidence and learning; • support the integration of learning from different parts of the course and beyond.

11. Outcome 1 Plan portfolio assessment in line with intended learning outcomes and course learning and teaching activities to enhance higher order learning

12. Planning aligned assessment methods Learning outcomes Teaching and Learning Activities Assessment methods Think and act like a biologist, historian, or computer scientist, etc. Act like a biologist, historian or computer scientist Learn to act like a biologist, historian or computer scientist Learning outcomes, learning activities and assessmentare tightly linked.

13. What is learning/higher order learning (HOL)? From: http://www2.warwick.ac.uk/services/cap/resources/pubs/eguides/eskills/guidelines/higher/

14. Aim: higher order learning • Base assessment tasks on using rather than replicating knowledge, e.g. assess through problem-solving which requires knowledge-based analysis and judgement • Assess in varied ways and include tasks (eg Modified Essay Questions - MEQs, projects) which require deep engagement and relate to real-life roles and competencies

15. Embed feedback and promote student understanding of how work is assessed Figure: Berry O’Donovan & Chris Rust ASKe Centre for Excellence in Teaching and Learning, (Assessment Standards Knowledge exchange) Oxford Brookes University

16. Outcome 2 Evaluate the potential impact of portfolios on student learning

17. A systematic review of the use of portfolios (in Burns 2008) Available evidence demonstrates that portfolios can support both the learning and assessment of general competencies ... • Success factors included: concurrent use in both formative promotion of learning and summative assessment; (mentor) • Summative assessment of the portfolio was important in ensuring portfolio learning maintained its status alongside other assessed material.

18. E-portfolios: flexibility (Woodward 2004)Digital portfolios: Fact or fashion? ‘One of the inherent dangers with digital portfolios, for example, is that the technological novelty of the product could overshadow the purpose of the portfolio. The danger is that learning to use the technology itself could then subsume the learning opportunities of portfolio construction.’ ‘There is strong evidence from this research that digital portfolios need to be developed within a carefully designed framework in the same way the paper-based portfolios have been developed …’

19. E-portfolios: flexibility (Woodward 2004)Digital portfolios: Fact or fashion? Cont … • …the combination of text, audio, graphic and video based representation of information collectively termed ‘multimedia’  student engagement in learning • Ownership of author and user: ‘hyper-linked portfolios offer choice to their audience’ (p.230) • ‘Hartnell-Young and Morris, however, caution that a “multimedia portfolio is not expected to be a graphic designer’s dream, the emphasis should be on learning” (1999, p. 28).’

20. E-portfolios: comparison three universities (Wilhelm 2006 p.70) • ‘Implementing e-portfolios helps to develop a “culture of evidence” (Barrett & Wilkerson, 2004) for ongoing program improvement. E-portfolios are a useful assessment tool in this process.’

21. Portfolios: reflection (Jones 2010) ‘… reflection is a process of critically examining one’s present and past practices as a means of building one’s knowledge and understanding in order to improve practice.’ ‘… a more comprehensive understanding of reflection was evident and many … reported that articulating their personal theory (often for the first time in their career) impacted positively on their ability to reflect on practice.’

22. Portfolio assessment and interview The effectiveness of the use of a portfolio is enhanced by combining portfolio assessment with interview. ‘a single-examiner portfolio interview focusing on standardised questions and a global rating scale is a feasible portfolio assessment method that can be used to assess clinical reasoning skills in an integrated, professionally authentic manner.’ Burch and Seggie (2008)

23. Outcome 3 Outline a plan of action: how could your students work with portfolios?

24. How to do portfolio assessment: practical steps • Devise assessment tasks which give students the opportunity to show that they have achieved intended learning outcomes; • Build in choice and drafting and redrafting with self- and peer assessment; • Set definite time limits for oral or filmed work, number of images and strict word limits for written work; stress quality above quantity.

25. How to do portfolio assessment: criteria • Write criteria for the assessment of the portfolio and give students the opportunity to mark each other’s work applying the criteria • Ensure that students know how marks will be awarded for each section of the portfolio and each aspect of the work

26. Portfolio assessment: an example from MSc oncology, UCL Presentation of a Portfolio of work developed through the year The portfolio will consist of6 written pieces covering all six modules: a. The Holistic Care assignment (details of which can be found later in this document) plus either: b. 4 case studies and a critical review or c. 3 case studies, a critical review and an audit report.

27. Positive outcomes for the student experience • The necessary acts of production, selection, critical judgement and reflection are, I believe, profoundly educational and developmental. (Baume 2001 p.11) • Students value [portfolios], as a tangible outcome from and demonstration of their learning (Baume 2001 p.19).

28. References Baume, D. (2001). A briefing on assessment of portfolios [Electronic Version]. Learning and Teaching Support Network Generic Series Assessment 6 from http://www.bioscience.heacademy.ac.uk/ftp/Resources/gc/assess06portfolios.pdf Biggs, J. (2003). Teaching for Quality Learning at University. 2nd ed. Buckingham: The Society for Research into Higher Education & Open University Press. Burch VC, Seggie JL. (2008) Use of a structured interview to assess portfolio-based learning. Medical Education, Vol. 42:894-900 Jones, E. (2010). Personal theory and reflection in a professional practice portfolio [Electronic Version]. Assessment & Evaluation in Higher Education, 35, 699-710 from http://dx.doi.org/10.1080/02602930902977731 Klenowski, V., Askew, S., & Carnell, E. (2006). Portfolios for learning, assessment and professional development in higher education. Assessment & Evaluation in Higher Education, 31(3), 267-286. Mentowski, M. and Associates (2000). Learning that lasts: integrating learning development, and performance in college and beyond. San Francisco: Jossey-Bass O’Donovan, B., Price, M., and Rust, C. (2004) Know what I mean? Enhancing student understanding of assessment standards and criteria. Teaching in Higher Education, Vol. 9, No. 3. Rees, C. and Sheard, C. (2002) The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience.Medical Education, Vol. 38, No. 2: 138-144 Wilhelm, L., Puckett, K., Beisser, S., Wishart, W., Merideth, E., & Sivakumaran, T. (2006). Lessons Learned from the Implementation of Electronic Portfolios at Three Universities [Electronic Version]. Tech Trends, 50 from http://www.springerlink.com/content/l3412700x44l4752/fulltext.pdf Woodward, H., & Nanlohy, P. (2004). Digital portfolios: fact or fashion? [Electronic Version]. Assessment & Evaluation in Higher Education, 29, 227-238 from http://dx.doi.org/10.1080/0260293042000188492

29. Links from LTSS: e-portfolioshttp://www.ucl.ac.uk/ltss-blog/?cat=74 http://www.ascilite.org.au/conferences/auckland09/procs/joyes.pdf http://www.jiscinfonet.ac.uk/infokits/e-portfolios/index_html