Can electronic portfolios facilitate personal development planning and meet the requirements for HE Progress Files and C

1. Simon Cotterill*, Tony McDonald*, Trudie Roberts^, Chris Roberts&, Margery Davis+, Geoff Hammond* Can electronic portfolios facilitate personal development planning and meet the requirements for HE Progress Files and CPD ? On behalf of the consortium “Managed Environments for Portfolio-based Reflective Learning” (http://www.eportfolios.ac.uk). * University of Newcastle upon Tyne ^ University of Leeds & University of Sheffield + University of Dundee Newcastle University

2. Overview • Background • Electronic portfolios • FDTL4 Project • Electronic Portfolio Approaches for HE Staff • Questions / Discussion !

3. Background: PDP • Increasing demands on HEIs to support personal development planning (PDP) • Recommendation 20 of the National Committee of Inquiry into Higher Education 1997 (Dearing Report) • “…institutions of higher education, over the medium term, develop a Progress File. The File should consist of two elements: • a transcript recording student achievement which should follow a common format devised by institutions collectively through their representative bodies; • a means by which students can monitor, build and reflect upon their personal development.”

4. Background: PDP • QAA recommendations on: • - transcripts • - progress files • PDP is primarily a formative process by which students can monitor, build and reflect upon their personal and academic development. • Built on the model of the adult / independent learner

5. Background: Internet Approaches to PDP • Internet Personal and Academic Records (PARs), Newcastle/Nottingham • - Two sequential projects funded by the DFES • LUSID (Liverpool), • RAPID (Loughborough), • PESCA (Exeter) • ProFile (Bangor). • Oxford Brookes / Thames Valley Joint Progress File • SHELL (Plymouth+)

6. 1st Internet-PARs project: • DfEE Recording Achievement Theme: • Internet-PARs Project (1998 - 2000) Partnership between Universities of Newcastle and Nottingham to:- … design, evaluate and disseminate a process, facilitated by C&IT, which would enable HEIs to construct PARs accessible on the Internet … to form the basis of a continuing record of lifelong reflective learning and personal development … demo via: http://www.internet-pars.ac.uk

7. Tutor’s view: • Academic • Support • Progress • Review • Development • Planning

8. The 2nd Internet-PARs project • Newcastle-Nottingham Internet-PARs Project Phase 2: Making New Links (2000 - 2002) • DfEE / DfES funded, HEFCE administered • … web-based systems to support continuing professional development (CPD) … • in Medicine (pre-registration house officers) • in Education (newly-qualified teachers) • … dissemination of experience of implementing both paper-based and web-based PARs at the institutional level … • a range of types of consultancy available to HEIs to facilitate planning and PDP implementation

9. communications • role based communication • employment • records of past, present & future posts • education • record courses, leave, & budget • development • plan, evidence, & reflect on experiences Overview of the web portal in Medicine Personal data MyPIMD Web portal Admin data (HR Database)

10. MyPIMD http://mypimd.ncl.ac.uk • Access controlled by login (establishes role and level of access). • Contents customised (current post, grade, specialty and location).

11. education

12. Encouraging recording & reflecting practice Facilitating mentoral processes Assessing learning needs and action planning development

13. Using admin (HR) and personal data to provide a chain of evidence for appraisals, assessments, revalidation… encouraging and facilitating on-going documentation and reflection on professional development

14. development Use of exemplification to encourage reflection

15. PORTFOLIOS • What are portfolios ? • FDTL4 project developing electronic portfolios

16. What are Portfolios ? • Unstructured • Formative • Sample • Best work • Factual / Quantitative • Student Owned Structured Summative All work Representative Reflective / Interprative HEI Owned

17. What is an Electronic Portfolio ? • A computer application which stores the user’s information on a local drive. • A static Web page eg. CV on the Internet • A dynamic database-driven systems that is accessed over a network.

18. My Portfolio Benefits of Electronic Portfolios • Multi-purpose • Organised • Multiple structures / views • Searchable • Transportable • Sharable • Not left on the bus !

19. Potential Weaknesses of Electronic Portfolios • There may be barriers to effective use of electronic portfolios: • Poor access to computing / Internet facilities • Technophobia ! • Reliability of IT systems • Possible duplication / overlap with paper systems It may also be important to use portfolios as part of a wider strategy to develop the reflective analytical skills necessary for PDP.

20. Learning Style - “natural reflectors”? Potential Weaknesses of Reflective Portfolios • It is unclear if reflective portfolios favour particular groups: • Gender and Age • Cultural Background • John Mole. The geography of thinking. • Clin Med 2002; 2:343-5

21. Formative and Summative Assessment • Portfolios come in many forms and are often used to evidence the achievement of learning outcomes for summative assessment. • There may be potential problems if the portfolio serves both formative and summative processes. • For example reflection and PDP is less likely to be open and honest if the student knows that the work will be assessed. More fundamentally the production of a portfolio can itself be a learning process ie. it is as much a journey as an end-point for assessment.

22. Formative and Summative Assessment • The use of electronic portfolios may potentially reduce the tension between formative and summative processes by supporting both assessed and non-assessed areas. • Students may also have the choice to make certain material available to assessors, appraisers, tutors and others involved in their education.

23. Students control which parts of their portfolio can be viewed for specific purposes or by specific people Data Ownership Model Personal Data Corporate (HR) data Tutor’s Data

24. Managed Environments for Portfolio-based Reflective Learning Integrated Support for Evidencing Outcomes.  • The project consortium: • Newcastle University (lead site) • Leeds University • Sheffield University • Dundee University (for consultancy) • 3 years funding through The HEFCE Fund for the Development of Teaching and Learning (FDTL). • Developing Web based portfolios to support reflective approaches for evidencing the attainment of programme outcomes in undergraduate Medicine. • Closely integrated with on-line curricula and study guides to become an integral component of managed learning environments for Medicine.

25. What is a managed learning environment ? • TLTP3-86: Facilitated Network Learning in Medicine & Health Sciences (Networked Learning Environments) • Newcastle University – Medicine • Nottingham University – Medicine • Durham University – Biomedical Sciences • Northumbria University – subjects allied to medicine • Sheffield University – Medicine • Study Guides Learning Resources • Module Outline Forms Self-assessment

26. Aims of the FDTL-4 project • To develop an on-line portfolio system to support a reflective approach to evidencing the attainment of programme outcomes. • To promote the development of the reflective capabilities of medical students, giving greater responsibility for managing their own learning and preparing for aspects of work-based and lifelong learning. • To develop strategies to facilitate assessment of curriculum outcomes that are not amenable to traditional instruments of assessment.

27. Managed Learning Environment(after JISC) Newcastle NLE curriculum map - outcomes delivery - campus and at distance learning resources self-assessment quality process tutor/mentor support off-line learning communication tracking registers student record system business systems other agencies other colleges Undergraduate portfolio and transcript for reflective learning - triggered by learning episodes related to outcomes

28. learning environment experience / practice experiment / apply review / reflect think / conceptualise communicate study Benchmarks  Outcomes DP Specs  Study Guides  Learning Resources experiential learning cycle

29. communicate study Benchmarks  Outcomes DP Specs  Study Guides  Learning Resources personal & academic record learning log evidence evidence learning experiential learning cycle

30. assess assess communicate study Benchmarks  Outcomes DP Specs  Study Guides  Learning Resources personal & academic record student transcript learning log evidence assess experiential learning cycle

31. communicate study Benchmarks  Outcomes DP Specs  Study Guides  Learning Resources personal & academic record student transcript learning log reflect evidence assess experiential learning cycle assess assess

32. Learning Outcomes The doctor as a professional Who? How? What? How the doctor approaches his/her practice What the doctor is able to do Doing the Right Thing When? Where? Doing the Thing Right The Right Person Doing It

33. Recording against terminal learning outcomes:

34. Defining intended learning outcomes for options:

35. Life Long Learning / CPD • Electronic portfolios can support life-long learning by drawing on personal and academic information prior to HE, such as the UCAS profiles. • The HE portfolios could later form the foundation for vocational or post-graduate portfolios, especially where portfolios are structured around learning outcomes which are geared towards key-skills and/or professional requirements.

36. Undergrad. Postgrad. 5yrs 10yrs The use of electronic portfolios aims to foster the skills of independent learning, reflection and PDP Life-long Learning Approach in Medicine: Pre-HE portfolios reflection / PDP -integrated into an outcomes based curriculum • continuing need to • demonstrate CPD: • annual RITA assessments • annual consultants appraisal • 5 yearly revalidation

37. Portfolios for HE Staff • Baume D, Yorke M. The reliability of assessment by portfolio on a course to develop and acredit teachers in higher education. Studies in higher education. 2002; 27: 7-25 • HE Staff may also benefit from electronic portfolio approaches including personal profiles and research plans which tie in to management information systems and networked learning environments.

38. The Personal Plan at Newcastle • Teaching and Learning (UG and PG teaching and teaching development) • Research (Research programmes, awards, staff and students) • Third Strand (spinouts, outreach work and industrial collaborations) • Roles (administrative duties, training and support roles) • Integrates centrally held data sets: • RIS awards data under "Research Grants held" • Graduate School records of PG student supervision under "Research Students".

39. Evaluation and Research Issues • Portfolio assessment – validity and reliability issues Enhancing reliability in portfolio assessment: discussions between assessors. Pitts J, Coles C, Thomas P, Smith F. Med Teach 2002; 24(2):197-201 The use of portfolios for assessment of the competence and performance of doctors in practice. Wilkinson TJ, Challis M, Hobma SO, Newble DI, Parboosingh JT, Sibbald RG, Wakeford R. Med Educ 2002: 36(10):918-24 Portfolio-based assessments in medical education: are they valid and reliable for summative purposes? Roberts C, Newble DI, O'Rourke AJ. Med Educ. 2002: 36(10):899-900 [Commentary] AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment Friedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJ Medical Teacher. 2001; 23:535-51 Portfolio assessment in medical students' final examinations. Davis MH, Friedman M, Harden RM, Howie P, Ker J, McGhee C, Pippard MJ, Snadden D. Med Teach 2001; 23(4):357-366 An evaluation of a standards based portfolio. [Nursing / Outcomes Based] Gallagher P Nurse Educ Today 2001;21(5):409-16

40. Evaluation and Research Issues • Reflective ability / practice • agreeing a context sensitive definition of “reflection” • qualitative analysis of anonymised portfolio data • based on this definition of reflection • quantitative analysis of anonymised portfolio data • - what factors influence reflective ability / practice ? • - gender, age-group, ethnic groups etc. • - any relationship with outcomes (grades) ?

41. Evaluation and Research Issues • Learning styles / learning approaches • - Honey & Momford used in study skills course • - ASSIST questionnaire • User evaluation • - questionnaires + focus groups • Student attitudes • questionnaires + focus groups (longitudinal) • effect of implementation techniques ?

42. Evaluation and Research Issues • Impact of portfolios on the Education Environment • Roff S. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Medical Teacher, 1997; 19: 295-299 • -Does the introduction of electronic portfolios change the educational environment ? • Other issues • Can computer mediated environments promote open reflection, sharing and sence of ownership ? • -Does the use of exemplification benefit or hinder reflection ?

43. Conclusions • Electronic portfolios have the potential to facilitate PDP and meet the requirements for HE Progress Files. • Offer flexibility for both formative and summative purposes. • Electronic portfolios in HE may provide the foundation for portfolios in CPD. • Transfer of data 17/18 HE CPD • Common outcome themes • Particularly valid where the curriculum is driven by professional requirements

44. Conclusions (2) • There remain many unanswered questions about the educational impact of electronic portfolios. • Importance of evaluation and research • Electronic portfolio approaches may also be applied to staff in HE for a range of purposes.

45. further information http://www.eportfolios.ac.uk http://www.internet-pars.ac.uk http://nle.ncl.ac.uk http://www.ltsn.ac.uk/genericcentre http://www.recordingachievement.org