Evaluating a transition ePortfolio

1. Evaluating a transition ePortfolio ePortfolio CETLs & DeL projects seminar University of Nottingham–July 20th 2006 Phil Harley 14-19 Transition Strategy Manager City of Nottingham Local Authority, UK

2. Functional Definition • A collection of authentic and diverse evidence (drawn from a larger archive) • Representing what a person/organisation has learned over time (after reflection) • Designed for presentation to one or more audiences for a particular purpose Process and Product

3. Key priorities for Passportfolio • Improving transition • Personalised Learning • Improved advice and guidance • Improving and tracking progression • Encourage partnership working • Recognising all achievement

4. Passportfolio • Learners • Achievement Zone • Reviewing Zone • Presenting Zone Teachers Information Zone Student Zone

5. L I F L O N G Achievement Zone Personal data Work Related Curriculum Work Experience Achievement Transcript Reviewing Zone Student Self Review CEG Programme Skills Assessment Individual learning plan L I F E W I D E Presenting Zone Application forms Transcript Curriculum Vitae Enquiry Application Induction

6. Passportfolio • Messaging service • Work – a multimedia online space • Diary – personal reflection area • Research space • Reminders of targets • Ask Aardvark advice & support function

10. Reviewing Zone STEP 7: Setting Goals STEP 6: Getting There STEP 5: Getting Started STEP 4: My Ambitions STEP 3: My Learning Styles STEP 2: My Skills STEP 1: About Me Individual Learning Plan

11. A checklist for schools and colleges • Technical capabilities e.g. storage • Interoperability with other services • Usability e.g. navigation • Possibilities for social interactions • Differentiated materials • Links with MIS • Can impact be measured

12. What could we evaluate? • Quality of applications • Effect on induction & retention • Social interactions • What it adds to the curriculum • Usability • Enjoyment, engagement & motivation • Raising self-esteem, self-awareness & empowerment • Its effect on learning

13. What we did evaluate? • Meeting stakeholder needs • Usability • Reviewing zone activities • The process of writing a personal statement • The personal statement • Levels of enjoyment • Levels of self-awareness

14. How did we evaluate? (1) Methods • Focus groups • Observation • Questionnaires • Pre activity • Activity • Post activity

15. How did we evaluate? (2) Four institutions • 11-16 city school – year 9 • 11-18 county school – year 10 • 11-18 city school – year 12 • FE college - level 2

16. Findings (1) • Perceived usefulness • 93% said e-portfolio would be useful and gave reasons • Usability • 88% felt confident with software

17. Findings (2) • Activities • 84% said they were enjoyable and 88% said they helped in writing a personal statement • Enjoyment • 86% enjoyed using Passportfolio whilst 57% found it stimulating!

18. Findings (3) • Self-awareness • 76% learned more about themselves using the e-portfolio • Personal statement • 91% were happy with personal statement and 95% felt it reflected the kind of person they were. 84% wanted examples

19. What they said (1) • “I like this because no one has ever asked me what I’ve achieved before” • “I realise that I have achievements even though it isn’t in school” • “It would be good for jobs, colleges and university applications” • “It would be useful to keep track of all my work and what I want to do”

20. What they said (2) • “I get to look over what I’ve achieved in life” • “It will store all my information I’II need for jobs or college” • “I’ve written down things about myself that I usually wouldn’t have thought of” • “I liked writing about myself because I never realised how much I have achieved” • “It made me look at myself”

21. Some difficulties in evaluation • Evaluation as part of the design process • Many other factors influence a personal statement • Impact will need to be determined by longitudinal studies • Decide what to evaluate, incorporate in project before use

22. Success factors • Identifying institutional priorities and offering to solve problems • Recognising cultural differences • Championing and enabling partnership and collaboration within an independent framework • Supporting the varying levels ofawareness and technical knowledge

23. Issues around future developments • Students need ULN early • E-portfolio as private and public document • Moving portfolio across and around institutions • Single sign on • Accessing qualification databases/verification • Links with e-assessment • Ownership issues • Transition data e.g. reports from schools and progression data back to schools • Linking to other web services

24. References • The original e-portfolio can be found at www.cityofnottinghampassport.com • Details of the Centre for International e-portfolio Development at Nottingham at www.nottingham.ac.uk/eportfolio • An article giving more detail on Passportfolio ferl.becta.org.uk/ • www.becta.org.uk • National policies at www.dfes.gov.uk • Definition at www.eife-l.org