Evaluating the Effectiveness of Problem Based Learning

1. DEE Conference – Cambridge 6-7 September 2007 Evaluating the Effectiveness of Problem Based Learning John Sedgwick and Guglielmo Volpe Department of Economics, Finance and International Business London Metropolitan University j.sedgwick@londonmet.ac.uk, g.volpe@londonmet.ac.uk Research funded and supported by the Economics Network of the Higher Education Academy www.economicsnetwork.ac.uk

2. Structure of Talk • Aims of Project • Evaluating Problem Based Learning: the literature • Structure of PBL Project • Evaluation of Experience • Evaluation of Performance • Students’ Perception • Staff Evaluation • Project’s Next Steps

3. Evaluating PBL: the literature Evaluation of Outcomes • Knowledge: mixed results or insignificant differences between PBL and other methods (Albanese and Mitchell, 1993); greater knowledge with PBL (Smits, 2002), (Farrell, 2003) • Lifelong Learning Skills: PBL superior to conventional methods (Bransford et al., 1989) • Learning Process: deeper approaches to learning ( Sobral, 1995) • Team Skills: PBL facilitates development of collaborative skills (Cockrell et. al., 2002) • Control Group Analysis: in some contexts PBL may lead to worse outcomes for some students (Newman, 2004)

4. Control Group Analysis • Structure of Project • Industrial Economics Module • Spring Semester Module • Taught at North and City Campus to two separate cohorts of students • PBL at North Campus while Traditional Approach at City Campus • Different lecturers in the two campuses • Same assessment but slightly broader syllabus at City campus • Control for students’ background and characteristics • Evaluate and compare students experience and performance

5. Methodology: evaluation framework (Newman, 2004) Control Group Analysis

6. Students’ Performance

7. Students’ Performance Does experience in PBL lead to a better performance? Regressor: Overall performance in module by North Campus studentsOLS method; R2=0.650, Adjusted R2 = 0.582; Observations: 38

8. Students’ Performance Is PBL correlated with a better performance in the module? Regressor: Overall performance in module by all studentsOLS method; R2=0.576, Adjusted R2 = 0.535; Observations: 58

9. Students’ Evaluation Students evaluation at end of semester

10. Students Evaluation – end of semester questionnaire

11. Students’ evaluation • Students’ views from questionnaires and focus group (PBL students) • “You are understanding while learning, as usually in lectures you do not retain the information” • “I hear and I forget, I see and I remember, I do and I understand” • “I believe with PBL students are forced to independently attain a wider knowledge because they need to research more and by so doing we come across more things than we would get in class teaching” • “PBL is good as it relates to real world issues and that tends to stay in my head. Group learning is also important as it is easier to understand a discussion than a lecture” • “PBL is not necessarily better. The tutor who is delivering the module and his ability to teach is important!” • “PBL is not good at all especially for final year students who need much more support at that time”

12. Staff Evaluation • “For me, these results are much stronger than I was getting in the module a few years ago, so I feel very encouraged” • “The experience was such a good one in terms of classroom buzz” • “I enjoyed the experience and will repeat it next year” • “The PBL experience induced me to introduce the approach in other modules”

13. Knowledge Retention • Both City and North Campus students were contacted after completion of moduleThey were asked to answer three questions without consulting any notesAttempts were marked blindly by both City and North campus tutors

14. Overall Evaluation

15. Steps to complete research… • Still two steps to complete research: • Full analysis of focus group with identification of key themes • Further evaluation of questionnaires by distinguishing responses by gender and age

16. Overall Evaluation • The PBL students perform better in the coursework but less well in the exam compared to the non-PBL students • Overall the non-PBL students have performed better but the difference in performance is not statistically significant • Students who experience PBL in the first semester perform better than those without such an experience • PBL is a positive but not significant determinant of performance across the two campuses • The majority of students appreciate the value of the PBL approach in particular for final year students • About half of the City students would have liked to experience PBL in the module • Students express some concerns about their ability to engage with PBL without having any prior experience • Staff enjoy the approach and the ability to be much closer to the students’ learning process