180 likes | 359 Views
Assessing verbal communication skills of medical students. J Voges E Jordaan * L Koen DJH Niehaus Department of Psychiatry, University of Stellenbosch and Stikland Hospital * Biostatistics Unit: Medical Research Council, Bellville. Positioning of the study. Large project:
E N D
Assessing verbal communication skills of medical students J Voges E Jordaan * L Koen DJH Niehaus Department of Psychiatry, University of Stellenbosch and Stikland Hospital * Biostatistics Unit: Medical Research Council, Bellville
Positioning of the study • Large project: • Correlation of communication skills with academic performance of medical students • Sub-studies: • Facial affect recognition • Oral examination marks in psychiatry • Non-verbal communication skills • Verbal communication skills
Introduction • Communication is one of six required competencies identified by the ACGME • Effective communication associated with: • Improved patient and doctor satisfaction • Treatment compliance • Strong predictor of medical school success • Assessment of communication skills is complex and difficult to implement
Aim First phase: • To evaluate the usability of the Liverpool Communication Skills Assessment Scale for assessing the communication skills of medical students of the University of Stellenbosch Second phase: • To determine effectiveness of undergraduate medical students’ communication skills using the Liverpool Communication Skills Assessment Scale • To determine if there is a correlation between communication skills and overall academic performance
Measurement equivalence • Central issue in determining the applicability of instrument cross-nationally and cross-culturally • Factors to consider: • Content equivalence • Semantic equivalence • Technical equivalence • Criterion equivalence • Conceptual equivalence (Flaherty et al, 1988)
Methods • Subjects: • Medical students completing late rotation • 5 min. semi-structured interview with patient that was videotaped • Permission granted by Faculty of Health Sciences and Ethics committee of SU • Venue: • 5-week Psychiatry rotation at Stikland hospital
Methods • Assessment tool: • Liverpool Communication Skills Assessment Scale (LCSAS) • Consists of 12-items measuring several aspects of communication • Mixed method of using both a checklist and a rating approach • 4-point ordinal rating scale ranging from Unacceptable to Good • Ease of use, acceptable reliability • Raters: • 2 independent raters, additional training • Third rater included, instruction given • Help sheet with additional descriptors to guide scoring • Primary statistical evaluation: • Inter-rater reliability • Marginal homogeneity (Chi-square statistic, p-value<0.01 as significant) • Agreement (Cohen’s weighted Kappa index for ordinal data)
Results: Distribution of score by rater • Intra-class correlation coefficient = 0.8 (0.71-0.87)
Agreement between raters • Agreement for items that had marginal homogeneteity (Cohen’s weighted Kappa index for ordinal data, 95% confidence interval)
Discussion • LCSAS evaluated for usability to measure communication skills in medical students • Additional training • Inclusion of additional rater and help sheet • Total score: high level of correlation • Inter-rater reliability • Marginal homogeneity • 4 of 12 items • Agreement • Additional training – greater agreement • Reliability of measure: • Continue with development and standardisation of assessment scale for use in South Africa • Training
Limitations and future directions • Limitations: • Small sample • Inter-rater reliability • Marginal homogeneity and agreement • Training • Future directions: • Re-evaluation of the scoring categories to promote understanding • Language • Gender • Culture
Conclusion • Use of LCSAS in South African setting • Correlation for total score • Training necessary to improve agreement for each item • Further development necessary • Use in education context of South Africa by various health professionals
Selected references • Epstein, R.M. Campbell, T.L., Cohen-Cole, S.A., McWhinney, I.R. & Smilkstein, G. (1993). Perspectives on patient-doctor communication. Journal of Family Practice 37(4): 377–388. • Flaherty, J.A., Gaviria, F.M., Pathak, D., et al. (1988). Developing instruments for cross-cultural psychiatry needs. Journal of Nervous and Mental Disorders176(5): 257-263. • Humphris, G.M. & Kaney, S. (2001). The Liverpool Brief Assessment System for Communication Skills in the Making of Doctors. Advances in Health Sciences Education 6: 69–80. • Parker, G. (1993). On our selection: predictors of medical school success. Medical Journal of Australia 158(11): 747–751. Project supported by funding from FINLO Faculty of Health Sciences