Loading in 2 Seconds...
Loading in 2 Seconds...
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Formative Evaluation of the Implementation of the Medical Education Metrics Standard for Continuing Education MedBiquitous Annual Conference 2011 May 11, 2011 Baltimore, Maryland
Presenters and Disclosures • Francis Kwakwa, MA Assistant Director, Data Management , Radiological Society of North America Does not have an interest in selling anything to CME professionals • Andrew Rabin, BS Chief Technology Officer, CECityDoes have an interest in selling a program or service to CME professionals • Sean Hayes, PsyD Vice President of AXDEV Group Does have an interest in selling a program or service to CME professionals • Valerie Smothers, MA Deputy Director, MedBiquitousDoes not have an interest in selling anything to CME professionals
Objectives of Today’s Session • Increase knowledge about use of Medical Education Metrics Standard (MEMS) in CME • Share experience of using MEMS as part of Program and Activity Reporting System • Present formative findings collected regarding the experience of those who have implemented MEMS • Offer case study findings as to how to assess implementation of new systems
MedBiquitous Metrics Working Group Mission To develop XML standards & Web services requirements and descriptions for the exchange of aggregate evaluation data and other key metrics for health professions education. 4
Technology standard for core evaluation data Users Educators want best practices, ability to compare Funders want to measure reach and efficacy Accreditors want to measure success of activity and provider MEMS: Medical Education Metrics
MEMS Data • Activity Description • What’s being evaluated • Participant Activity Evaluation • What did participants think • Participation Metrics • How many people participated • Learner Demographics
Program and Activity Reporting System (PARS) • Manual data entry • Comma delimited upload • XML upload 7
Formative Evaluation • Provides immediate feedback to program participants • Fosters reflection and self-regulation • Provides feedback about the program itself to designers and sponsors • Oriented to improving the focus of the program • Based upon a dynamic model
Relationship Between Formative and Summative Evaluation Across Life of an Innovation Formative Evaluation Relative Emphasis Summative Evaluation Program Life Worthen 1987
Focus Group • MEMS implementers using XML in association with PARS • 4 organizations represented • Measured • Success and satisfaction • Resources for implementation • Characteristics of implementers • Decision factors 10
Results • Implementers felt satisfied, but limited (not many customers implementing) • Limited effort and resources required; change management important • Implementers all fluent in XML • ACCME adoption key; implementers seeking to better support customers, show competitive advantage, make links between different types of data
Case Study • Implemented MEMS for PARS upload • Experience implementing MedBiquitous standards • Integrated new data into user interface and workflow • Provided user training
Lessons Learned • Standards implementation can bring about efficiencies when integrated into workflow and properly supported • Validation tools needed • Web services needed to realize full benefits, one click data transfer
Contact Us • firstname.lastname@example.org • Valerie will forward questions to the working group