An Integrative, Multi-Disciplinary Review of Small-Group Collaborative Learning

1. An Integrative, Multi-Disciplinary Review of Small-Group Collaborative Learning Prepared for the SDRME Summer Meeting July 22, 2013 Anna T. Cianciolo, PhD Southern Illinois University School of Medicine

2. Background: Does Problem-Based Learning Work? • 4 of the top 10 most-cited articles in medical education journals were reviews of PBL effectiveness (Rotgans, 2012) • Conclusion: PBL isn’t worth the hassle (Colliver, 2000) • No real difference on nearly all measures of academic performance • Studies finding positive effects had design flaws • PBL lacks evidence-based theoretical underpinnings, which should be developed and tested (Colliver, 2002)

3. Background: Critiques / Reactions • Curriculum-level evaluations lack the control necessary to detect cause-effect relationships; (Norman & Schmidt, 2000; Norman, 2003) • Is reductionism really the answer? • Too much emphasis is placed on short-term knowledge testing (Strobel& van Barneveld, 2009) • But if this is what board examiners look for… ? • Differential attrition masks the positive effect of PBL (Schmidt et al., 2012) • Is that good??

4. What Is Everyone Talking About? • Maudsley (1999) • PBL means different things for different institutions • Hak & Maguire (2001) • PBL tutorial sessions remain a black box • Svinicki (2007) • Cooperative learning is central to PBL, but unstudied • Dolmans & Gijbels (2013) • “…when implementing or investigating PBL, much more attention should be paid to the conditions under which the effects of PBL can be maximized… what should be considered as PBL and what [should be considered] otherwise?… important information about the details of the learning environment is often lacking.”

5. The Present Study • Comprehensive interdisciplinary literature review • Research award from the Society of Directors of Research in Medical Education (SDRME) • 2-year investigation • Define, functionally, what constitutes PBL • What must happen during a collaborative learning tutorial for the objectives of PBL to be reached? • Emphasis on process vs. characteristics • Identify the properties of the learning environment that promote/constrain individual learning in groups • Provide recommendations for implementing and evaluating a small-group collaborative learningcurriculum

6. Why Should We Care? • PBL is one type of small-group collaborative instruction; other types have similar implementation challenges associated with student-centered learning (Cianciolo et al, 2011) • Similar issues arise in other (related) areas • Computer-assisted instruction: “No significant difference”(Russell, 2001) • Scaling educational interventions (Yeager & Walton, 2011) • Innovations in medical education face the same fate • Does team-based learning work? • Do longitudinal clerkships work? • Does interprofessional education work? • Does simulation-based training work? • Etc.

7. Method (Cook & West, 2012) • Searched for eligible studies • Examined first- and second-tier journals outside medical education (avg. impact factor 2.91, range 1.15 – 6.72) • Example search terms: “group AND learning” “social AND learning” • Explored reference sections of articles selected • Followed-up with Google Scholar search including medical education articles • Exclusion criteria • Studies done with K-12 children • Studies done in non-Western cultures • Studies of simple laboratory tasks or motor/coordinative tasks • Studies focused specifically on computer-assisted learning or non-interactive groups • Studies lacking methodological rigor (quantitative or qualitative)

8. Method, Contd. • Abstracts were collected and reviewed, erring on the side of inclusion • N = 189, 31 excluded (so far) upon reading • Annotated bibliography made of articles selected • Working summary of results drafted and modified continuously over the course of the review until thematic saturation is reached

9. Findings: PBL Objectives* • Higher-order thought processes • Reasoning, critical thinking • Organized, actionable knowledge • Self-direction • Self-assessment and reflection • Motivation for learning • May be generalized across social learning approaches (Hmelo-Silver et al., 2007) • The raison d’etre of group instruction (Palinscar, 1998; Schmidt, 1993) • Interaction among learners exercises the cognitive activity necessary to achieve higher-order thinking and self-direction *Barrows (1986), Norman & Schmidt (1992)

10. Findings: The “Signature” of Effective Small-Group Instruction • Much more than the use of realistic, ill-structured problems discussed in a group setting with a facilitator • Questioning and Self-Explanation • Recognition of knowledge gaps and errors • Development of anticipatory reasoning, principle-based explanation (discovery of abstractions) • Exercise of self-regulated learning (via self-testing and assessment) • Associated with long-term retention and transfer of learning • Detectable in ongoing discourse and in essay responses and therefore an assessable indicator of small-group instruction quality • Enables active engagement by faculty without dominating the discussion

11. Interaction and Interdependence • Interdependence can be an important driver of interaction and the quality of group solutions (Day et al., 2005; De Dreu, 2007; Maloof & White, 2005) • Learner interdependence • Formal • Explicit, external control measures • Individual is held accountable for the performance of the group • Informal • Implicit control measures • Interaction is motivated by identification with and social obligation to the group

12. Characteristics of Formal Interdependence • Contractual in nature • High-stakes group assessment • Group problem has a clear deliverable and requires interaction to solve effectively • Clearly articulated, largely non-overlapping roles and responsibilities

13. Formal Interdependence in Undergraduate Medical Education? • High-stakes individual assessment • Group problems (e.g., cases) typically do not have a clearly defined deliverable • Interaction when solving group problems may or may not be necessary and is not formally assessed • Group member roles and responsibilities are unstated and completely overlapping • Interdependence must be fostered informally

14. Factors that Promote Informal Interdependence • Small group size (~6 or fewer) • Associated with lower levels of felt dispensability and higher cognitive effort • Low group diversity (ability, attitudes, race/ethnicity, age, social characteristics, etc.) • May increase group identification and decrease felt dispensability • Group teamwork knowledge and skill • Promotes communication, norms for cooperation, etc. • Promotes reflective practice and continuous improvement • Reduces the impact of diversity

15. Informal Interdependence in Undergraduate Medical Education? • Impact of group characteristics is weaker when the group is facilitated effectively (e.g., Donohue, 2001) • Several factors limit effective group facilitation • Teamwork knowledge and skill is not explicitly taught to faculty or students, nor is it effectively assessed • Faculty buy-in to group learning is idiosyncratic • Group characteristics seem quite likely to influence learning outcomes in medical school

16. Implications for Small-Group Educational Research • Curriculum-level evaluations must assess small-group effectiveness using questioning and self-explanation as criteria • “Traditional” methods also should be assessed with these criteria • Questioning and self-explanation may occur in other ways, which should be specified and assessed • Curriculum-level evaluations must specify the interdependence factors featured in all curricula being compared

17. Evaluation Assessment Targets Lecture Practices Knowledge Gap, Error Discovery Formal Interdependence Higher-Order Thought Questioning and Self-Explanation Abstraction, Principle Discovery Interaction Self-Direction Informal Interdependence Self-Testing, Self-Assessment Individual Study Practices

18. Implications for Curriculum Design and Implementation • Continuous faculty and group-leader development is essential as long as educational conditions lack formal interdependence • When small-group instruction is used, teamwork knowledge should be formally developed and assessed • Group-level assessments should be included in the formal evaluation of student performance to promote interdependence

19. References • Working bibliography is 85 references long and counting… • Please don’t hesitate to contact me for the complete list: acianciolo@siumed.edu