1 / 17

MedBiquitous Annual Conference 2011 May 11, 2011 Baltimore, Maryland

Formative Evaluation of the Implementation of the Medical Education Metrics Standard for Continuing Education. MedBiquitous Annual Conference 2011 May 11, 2011 Baltimore, Maryland. Presenters and Disclosures. Francis Kwakwa , MA

gareth-kaufman
Download Presentation

MedBiquitous Annual Conference 2011 May 11, 2011 Baltimore, Maryland

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formative Evaluation of the Implementation of the Medical Education Metrics Standard for Continuing Education MedBiquitous Annual Conference 2011 May 11, 2011 Baltimore, Maryland

  2. Presenters and Disclosures • Francis Kwakwa, MA Assistant Director, Data Management , Radiological Society of North America Does not have an interest in selling anything to CME professionals • Andrew Rabin, BS Chief Technology Officer, CECityDoes have an interest in selling a program or service to CME professionals • Sean Hayes, PsyD Vice President of AXDEV Group Does have an interest in selling a program or service to CME professionals • Valerie Smothers, MA Deputy Director, MedBiquitousDoes not have an interest in selling anything to CME professionals

  3. Objectives of Today’s Session • Increase knowledge about use of Medical Education Metrics Standard (MEMS) in CME • Share experience of using MEMS as part of Program and Activity Reporting System • Present formative findings collected regarding the experience of those who have implemented MEMS • Offer case study findings as to how to assess implementation of new systems

  4. MedBiquitous Metrics Working Group Mission To develop XML standards & Web services requirements and descriptions for the exchange of aggregate evaluation data and other key metrics for health professions education. 4

  5. Technology standard for core evaluation data Users Educators want best practices, ability to compare Funders want to measure reach and efficacy Accreditors want to measure success of activity and provider MEMS: Medical Education Metrics

  6. MEMS Data • Activity Description • What’s being evaluated • Participant Activity Evaluation • What did participants think • Participation Metrics • How many people participated • Learner Demographics

  7. Program and Activity Reporting System (PARS) • Manual data entry • Comma delimited upload • XML upload 7

  8. Formative Evaluation • Provides immediate feedback to program participants • Fosters reflection and self-regulation • Provides feedback about the program itself to designers and sponsors • Oriented to improving the focus of the program • Based upon a dynamic model

  9. Relationship Between Formative and Summative Evaluation Across Life of an Innovation Formative Evaluation Relative Emphasis Summative Evaluation Program Life Worthen 1987

  10. Focus Group • MEMS implementers using XML in association with PARS • 4 organizations represented • Measured • Success and satisfaction • Resources for implementation • Characteristics of implementers • Decision factors 10

  11. Results • Implementers felt satisfied, but limited (not many customers implementing) • Limited effort and resources required; change management important • Implementers all fluent in XML • ACCME adoption key; implementers seeking to better support customers, show competitive advantage, make links between different types of data

  12. Case Study • Implemented MEMS for PARS upload • Experience implementing MedBiquitous standards • Integrated new data into user interface and workflow • Provided user training

  13. Lessons Learned • Standards implementation can bring about efficiencies when integrated into workflow and properly supported • Validation tools needed • Web services needed to realize full benefits, one click data transfer

  14. Contact Us • vsmothers@jhmi.edu • Valerie will forward questions to the working group

More Related