Scott M. Strayer, MD, MPH Associate Professor Department of Family Medicine

1. Technology-based education and intervention tools in medical education: Lessons and methods for the University community Scott M. Strayer, MD, MPH Associate Professor Department of Family Medicine University of Virginia Health System

2. My Colleagues and Co-Authors • Sandra Pelletier, PhD • Crista Warniment, MD • Karen Ingersoll, PhD • Steve Heim, MD • John Schorling, MD, MPH

3. Preventable Causes of Death • The most common causes of disease, disability, premature death, and health care burden in the US can be directly attributed to 4 health risk behaviors: • Smoking tobacco • Risky use of alcohol • Unhealthy diet • Physical inactivity 1. McGinnis JM, Foege W. Actual causes of death in the United States. JAMA. 1993;270:2207-2212. 2. Mokdad AH, Marks JS, Stroup DF, Gerberding JL. Actual causes of death in the United States, 2000. JAMA. 2004;291:1238-1245.

6. Health Behavior Counseling • Obesity • In surveys of patients who had a routine checkup with a physician in the past 12 months, only 14% of respondents reported receiving advice from their physician to lose weight, even though approximately 34% of the sample was classified as overweight and 19% as obese (Sciamanna et al., 2000). • Moreover, only 16% of overweight individuals and 40% of obese individuals reported receiving advice from their physician to lose weight during their checkup within the past twelve months (Loureiro, 2006).

7. Health Behavior Counseling • Alcohol Use • Only 23% of binge drinkers are counseled about alcohol use (Am J Prev Med 2003;24(1):71–74) • A survey of family physicians and internists found that only 64.9% of respondents screened 80-100% of their patients for alcohol abuse or dependence during the initial visit and a mere 34.4% screened that many patients during an annual visit • Only 20% said treatment resources for early problem drinkers was adequate • 72% preferred not to counsel patients themselves (Spandorfer & Turner 1999)

8. Health Behavior Counseling • Smoking Cessation • Smoking cessation counseling only occurs at 23 to 46% of primary care visits • Only 35% of physicians assist with smoking cessation attempts • Less than 10% arrange follow-up for smoking patients • Thorndike AN, Rigotti NA, Stafford RS, Singer DE. National patterns in the treatment of smokers by physicians. JAMA 1998; 279:604-608 • Lewis CE, Clancy C, Leake B, Schwartz JS. The counseling practices of internists. Ann Intern Med 1991; 114:54-58. • Goldstein MG, DePue JD, Monroe AD, et al. A population-based survey of physician smoking cessation counseling practices. Prev Med 1998; 27:720-729.

9. Barriers • Physicians • Competing demands • Conflicting recommendations • Lack of training • Patients • Lack of knowledge • Fear of discomfort • Cost • Office • Poor reimbursement • Lack of systems

11. Opportunities for Intervention • Most people visit a primary care doctor about three times per year. • Even 2-3 minute interventions are effective, especially when followed up with telephone, e-mail, nurse calls, referrals, 1-800 numbers, etc. • Many primary care providers provide 2-3 minute health promotion/behavior interventions at every outpatient visit. Stange, KC, Woolf, SH, Gjeltema K. One minute for prevention: The power of leveraging to fulfill the promise of health behavior counseling. Am J Prev Med, 2002; 22:320-323.

12. Integrating the Behavioral Theories Strayer SM, et al. Development and evaluation of an instrument for assessing brief behavioral change interventions. Patient EducCouns (2010), doi:10.1016/j.pec.2010.04.012

13. Research Design • RCT with 3rd year family medicine clerkship students • All students received smoking cessation workshop based on 5 A’s, Stages of Change and MI • Randomization by paired rotation blocks to receive either a reminder card or a handheld computer tool - the Educational Smoking Mobile Intervention Tool (E-SMOK-I.T.) • Students assessed pre and post study for smoking cessation counseling knowledge and self-reported behavior and comfort using a previously validated survey instrument • Students assessed for appropriate smoking cessation counseling using video-taped standardized patient interview using an instrument that was developed to assess combined smoking cessation counseling interventions • 2 independent observers assessed key smoking cessation counseling activities using videotapes and rating form • Focus groups held with E-SMOK-I.T. tool users post clerkship

14. Study Flow Diagram

15. Educational Smoking Intervention Tool (E-SMOK-I.T.) Main Screen Assessment Screen

16. E-SMOK-I.T. (cont). Assist Screen Arrange screen

17. Analysis • Frequency and descriptive analyses • Inter-rater reliability for videotape assessment • Pre/post survey analysis using mixed ANOVA • Smoking cessation counseling skills using Independent t-tests

18. Results • 116 students completed the study and had complete video observations (N=63, control, N=53 intervention). • Mean age for participants was 26 and 52% were male. • 62% reported handheld computer literacy as intermediate on a 3 point scale. • Average Inter-rater reliability was 0.82 (Intraclass correlation)

19. Results • Overall smoking cessation counseling behaviors, knowledge and comfort increased among all participants (p<.001) • Paper-based group performed better than the handheld tool group (69% vs 62%; t=2.318, p=.022) at end of clerkship • No difference between groups at end of academic year (61% vs. 59%; t=.621, p=.538) • Focus groups of each rotation block revealed several possible reasons for results.

20. Pre-post Survey Results Knowledge pre/post increase 0.71 (F=15.54, p<0.001) Behavior pre/post increase 7.026 (F=194.45, p<0.001) Comfort pre/post increase 5.93 (F=163.620, p<0.001)

21. Counseling** p<.022, MI Spirit** p=.004, Retention p=.538 Smoking Cessation Counseling Skills

22. Focus Group Themes • Discomfort using at point of care or patient not receptive: • “I think I might have used it more if I were more comfortable with all the other aspects of being in that room.” • “… but the patients were either like I don’t want to quit or they knew what they needed.” • Forgot or Gossip about tool: • “I was afraid of it … I heard that there were problems with it with info-retriever I was afraid about that”

23. Focus Group Themes • Would have used if more review / training: • “It might be useful if I spent more time learning how to use it.” • Not enough time, smoking not priority: • “The times that it really gets brought up it is a long appointment you don’t have enough time to spend on smoking” • Tool used as educational reference • “I did like how you could go to related things inside the program. But mostly I went through it to review it.” • “Looked at it once or twice but not with patient”

24. Conclusions • Smoking cessation counseling by 3rd year medical students was improved using a combination of workshop and a supplementary reference tool. • Our hypothesis that the point of care tool would enhance skills significantly did not occur (unlike with our studies in practicing physicians) • Barriers specific to the 3rd year training period need to be addressed in order for a point of care tool to be effective in this setting

25. SBIR Phase 2 Contract

26. Phase II Research Aims • Conduct user-centered design evaluations to validate design and usability of new functionality • Assess the ability of QuitAdvisorMD to influence smoking cessation as measured by • The increase in clinician initiated smoking cessation counseling • The increase in ability of clinicians to provide appropriate smoking cessation counseling based on the PHS guidelines (and in particular, physician performance of the 5 A’s of: asking patients about smoking status, advising patients to quit smoking, assessing patient readiness to quit smoking, assisting patients with their quit attempts or in giving motivational interventions if they are not ready to quit smoking, and arranging follow-up for all patients who are smokers) • The increase in patient quit attempts.

32. Cross-study Themes • Content is a valued reference/resource (AMIT, QA, ACS) • Users liked the software as an educational tool (AMIT, ACS) • Need more practice and training (ACS, AMIT, QA) • Discomfort using computers with patients (AMIT, ACS) • Lack of time (ACS, QA, AMIT)

33. Conclusions • The content for computer-assisted counseling tools was appealing to all levels of learners • Many users see this software as an educational tool • All levels of learners requested more training and practice • Discomfort using computers with patients affects learners and even some practicing physicians • Lack of time for counseling is a persistent barrier

34. Topics for Discussion • Should health information technology tools (e.g. electronic medical records) be evaluated with learners as well as practitioners for official certification? • What are the best methods for evaluating health information technology tools with learners? • How would you feel as a patient if you were being interviewed by a medical student or resident using one of our tools described in our studies?