Evaluating the Case for Employing E-Portfolio Tools During the Final Year Engineering Project

1. Evaluating the Case for Employing E-Portfolio Tools During the Final Year Engineering Project Ray Sheriff and Felicia Ong

2. Contents • The Final Year Engineering Project • Process and Technology • Could e-Portfolio tools be applied to the FYEP? • Qualitative and Quantitative Research • Results • Supervisors and Students • Final Year Project: Skills and Personal Reflectivity (FYP:SPA) • A PebblePAD Application • Conclusion • What have we Learnt?

3. The Final Year Engineering Project Process and Technology

4. The Project Process • The Final Year Engineering Project (FYEP) requires students to apply a range of personal, practical and technical skills • This represents a new way of working for many students • At Bradford, the FYEP spans two semesters • Equivalent to 300 hours or 30 credits • Progress tends to be gauged through regular face-to-face meetings between supervisor and student • Provides opportunity to: • Assess progress • Monitor engagement • Discuss Technical Issues • Assessment occurs at the end of the project by: • Poster and interview (20%) • Concluding technical report (80%)

5. The Adoption of Technology • Technology in higher education is increasingly being used to enhance many aspects of learning and teaching, for example: • PowerPoint in the lecture theatre is now commonplace • Learning Management Systems such as Blackboard are widely used throughout HE • The adoption of technology is being driven by factors such as: • Increased availability of broadband Internet communications • Enhanced capabilities of Web tools and applications • Widespread take-up of affordable, easy to use electronic devices

6. E-Portfolio Tools • e-Portfolio Tools make use of the Web to: • Develop a portfolio of skills • Facilitate communication between individuals • Provide a means of record keeping • e-Portfolio tools have the possibility to enhance the FYEP experience from the perspectives of both student and supervisor • To date, e-portfolio tools have primarily been employed for personal development planning • University of Bradford provides all students with access to an e-portfolio tool (PebblePAD) for the duration of their studies

7. Aims and Objectives • Aim • To determine whether e-portfolio tools can be used to enhance the FYEP experience • Obj-1 • To identify existing practice among supervisors and students • Obj-2 • To raise level of awareness of capabilities of e-portfolio tools • Obj-3 • To evaluate the performance of e-portfolio tools when used for FYEPs

8. Could e-Portfolio tools be applied to the FYEP? Qualitative and Quantitative Research

9. Methodology • Apply a mixture of qualitative and quantitative techniques • Consult both supervisors and students • Perform initial data gathering through on-line questionnaires and semi-structured face-to-face interviews • Conclude with a pilot phase involving PebblePad, the University’s e-portfolio tool

10. Methodology: Qualitative Data • Data gathered through semi-structured face-to-face interviews with project supervisors • Eight interviews were conducted over seven weeks during the summer of 2011 • All interviewees were members of academic staff from the School of Engineering, Design and Technology (SoEDT) • The interviews, which were recorded and transcribed, followed a common structure covering: • Background and Experience • Meetings with Students • The Role of Technology • Project Skills and Assessment

11. Methodology: Quantitative Data • Data were gathered through on-line supervisor and student questionnaires • Generated using the Google Docs application • Available over the final two weeks of May 2011 • Questionnaires were divided into four sections: • Background • Meetings • Technology and the FYEP • Project Skills and Assessment

12. Example On-Line Questionnaire Pages Student Supervisor

13. Results Supervisors and Students

14. Semi-Structured Face-to-Face Interviews • 234 minutes of recordings were made with FYP supervisors • An interview on average lasted 29.25 minutes • 150 most commonly occurring words

15. Sample Quotes – Members of Academic Staff • “So every week I meet with all my project students at the same time and every student gives a short presentation about the work they have done the week before, ...” • “I think it is very much easier to gauge where the student is in a face-to-face meeting and to respond to particular technical issues, to explain complicated points and so on …” • “I think one thing that probably is definitely missing ispresentation skillsor public speaking skills especially when we are trying to assess them on posters and on how confident they were and how they answered …” • “I suspect PebblePAD might be useful but again I have not got a clear vision of how it is supposed to be used and how it might be implemented.” • “if they are using the log-book, there is at least some written indication of what they have done.”

16. On-Line Questionnaire: Supervisors • Thirteen of SoEDT’s academic staff completed the on-line questionnaire • Overall, a response rate of 27% was achieved • Just over half of the academic staff that completed the on-line survey had been supervising FYEPs for more than 10 years • None of the academic staff had been supervising students for fewer than 3 years • More than 60% of academic staff supervised between 5 and 6 FYEP students each year • The minimum number being between 3 and 4 • All bar one of the respondents indicated that e-mail was their main means of communicating with students

17. Supervisors – Sample Results

18. Observations - Supervisors • Most met with students on a weekly basis • Where would students benefit from Training? • 90% agreed that Time Management was important • Other areas to receive good support included Project Management (84%), Research Methods (76%) and Presentation Skills (76%) • Familiarity with Technology • Limited expertise in a number of packages • Most supervisors are at least competent in the use of the University’s Learning Management System (LMS) package Blackboard • Other applications such as Facebook, as well as the e-portfolio tool PebblePAD have very little exposure • Project Practice • No strong opinion on whether Personal Development Planning (PDP) would add value with 46% of returns agreeing and 30% disagreeing • Adding a reflective account gained some support with 61% in favour • Maintaining log-books as part of good practice had strong support (84%)

19. On-Line Questionnaire: Students • 31 students completed the survey • Corresponds to a response rate of roughly 13% • Mechanical and Medical Engineering had the greatest student representation (39%) • Followed by Electronics and Telecommunications (26%)

20. Students – Sample Results

21. Observations - Students • Most met with their supervisors on a weekly basis • Where would students benefit from training? • 90% indicated an interest in Research Methods • Other areas to receive good support included Report Writing (86%), Presentation Skills (83%), Time Management (80%) and Project Management (80%) • Familiarity with Technology • High level of expertise in the University’s LMS, Blackboard, with 92% being at least competent • Students indicated at least competence in Facebook (70%) and Skype (73%) • PebblePAD was untried by 87% of returns • Project Practice • Supportive of the use of PDP as part of the FYEP experience, with 80% agreeing that this would add value to the project • Reflective writing also received good support with more than 75% in agreement • Maintaining log-books as part of good practice was supported by 90%

22. Final Year Project: Skills and Personal Reflectivity (FYP:SPA) A PebblePAD Application

23. Pilot Trial Phase • An application, Final Year Project: Skills and Personal Reflectivity (FYP:SPA), was developed to conclude the study • Based on an existing PebblePAD application developed for 1st Year Induction • SaPRA: Skills and Personal Reflective Activity • FYP:SPA allows students to self-evaluate their competence on a scale of 1 to 5 under various activities, clustered under six Skill Statements: • Academic Writing • Academic Reading • Communication and Presentation • Individual and Learning Strategies • Library and Referencing • Research • Resources within each skill set are used to inform students of freely available training opportunities • As a student undertakes training and collects evidence, self-evaluation can be re-performed to demonstrate personal development and learning progression • The student then has the option to share this development in the form of a portfolio of work with their supervisor

24. Example Skill Statement: Academic Writing

25. Concluding Remarks What have we Learnt?

26. Conclusion • The FYEP represents a new way of working for most students • During the course of the FYEP, regular meetings occur between supervisor and student • E-mail is the main means of contact outside of meetings • E-portfolio tools facilitate communication and record keeping • The use of log-books has strong support from supervisors and students • There are barriers to be overcome if e-portfolio tools are to be considered relevant to today’s FYEP experience • The questionnaires illustrated a lack of awareness of the capabilities of the University’s e-portfolio tool, PebblePAD, among supervisors and students • A lack of exposure to the potential benefits of e-portfolio tools will need to be addressed if FYEP usage is to move from early adoption to mainstream • The FYP:SPA application offers the opportunity to introduce a structured approach to training and personal development • The developed approach could equally be applied to other disciplines or to other levels of study

27. Contact Details • For further information please contact: • Ray Sheriff • 01274 234053 • r.e.sheriff@bradford.ac.uk • Felicia Ong • 01274 236262 • f.l.c.ong@bradford.ac.uk

28. Acknowledgement • This presentation is based on work that was performed as part of the National HE STEM Programme