1 / 20

Basing Practices on Your Own Evidence: Elevate Your Data

Danny Singley, Ph.D. Director of Curriculum Development and Research Essential Learning Carol Hurst, Ph.D., LCSW Clinical Quality Enhancement Instructor and Coach Corporate University of Providence Providence Service Corporation. Basing Practices on Your Own Evidence: Elevate Your Data.

shaw
Download Presentation

Basing Practices on Your Own Evidence: Elevate Your Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Danny Singley, Ph.D. Director of Curriculum Development and Research Essential Learning Carol Hurst, Ph.D., LCSW Clinical Quality Enhancement Instructor and CoachCorporate University of ProvidenceProvidence Service Corporation Basing Practices on Your Own Evidence: Elevate Your Data

  2. Overview • Do’s and Don’ts for constructing surveys • Setting up custom surveys in Elevate • Choosing the report format that’s right for you • Results from a recent research project conducted with Providence

  3. Surveys • Uses in organizations • Who develops them • Data collection methods • Disseminating and acting on results

  4. Developing Surveys – Do’s • Chefs in the kitchen • Keep the outcome in mind • Use existing forms and data • Items -less is more • Meaningful sub-scales • Standardize the items and response mode • Balanced positive/negative frame • Incentivize and follow up

  5. Developing Surveys – Don’ts • Too many items – respondent fatigue • Messy/Non-uniform item types • All qualitative responses • “Neutral” responses • “Double-barreled” items • Negatively-framed items • Burying results • Going beyond the results

  6. Surveys in Elevate • Post-tests vs. surveys • Create a “sham course” • Test it • Create survey curriculum • Assign to respondents • Sit back and let the data roll in • Reports

  7. Reporting • Reports • Aggregate data • Raw data • Intended audience • Sound bites and pictures

  8. E-Learning Effectiveness Research Early Research Findings (1994-2006) • E-Learning and face-to-face equally effective DoE’s (2009) Meta-Analysis – 51 studies • Both pure and blended online learning are superior to face-to-face • E-learning enhances learning with more time and reflection exercises • Blended training more effective than pure online training when compared with face-to-face

  9. Our Training Effectiveness Study Addressed how learners 159 Providence clinicians benefited from taking the same five module course -Making Parenting Matter: Coaching Parents on Positive Parenting - in four different conditions: • Live Workshop – one day (n = 46) • Tele-class - five facilitated weekly conference calls (n = 46) • E-learning - five weekly e-learning courses (n = 45) • Waitlist group (n = 22)

  10. The MPM Survey • Developed for this study • Face valid • 43 items pre-test • 36 items post-test and follow-up • Four subscales: Applicability, Understanding Self-Efficacy, Utilization

  11. Goals and Hypotheses of the Study Goals • Evaluate the different modes of training • Compare ROI for different modes • Incorporate EL Connect into learning • Use Elevate for data collection Hypotheses • Intervention groups will show improved training outcomes • Tele-class and e-learning will show greatest ROI

  12. MPM E-Learning Course: Meet Carol’s Avatar l

  13. Social Networking on ELC

  14. Results - Training Effectiveness

  15. Results - ROI

  16. Attrition • Some participants dropped out • More attrition in e-learning and tele-class conditions compared to workshop • Themes from qualitative feedback: - Technical difficulty - Job demands - Preference for face-to-face

  17. Implications • E-learning, tele-classes, and face-to-face showed comparable learning outcomes as compared with the control group. • E-learning and tele-class trainings are considerably more cost-effective than face-to-face workshops.

  18. References Dillman, D.A. (2007). Mail and Internet Surveys: The Tailored Design Method, Wiley: Hoboken, New Jersey. Sudman, S., Bradburn, N.M, & Schwartz, N. (1996). The Application of Cognitive Processes to Survey Methodology, Jossey-Bass, San Francisco. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies, Washington, D.C. www.ed.gov/about/offices/list/opepd/ppss/reports.html

  19. Summary • Take care to design useful surveys • Elevate is a key data collection resource • Match your reports to your audience • E-learning and tele-classes were as effective than face-to-face workshops in terms of learning outcomes • E-Learning was the most cost-effective of all the training modalities in this study

  20. Go Forth and Collect Data Thank you very much!! For more information, contact: Dr. Danny Singley dsingley@essentiallearning.com Dr. Carol Hurst churst@provcorp.com

More Related