Basing practices on your own evidence elevate your data
Sponsored Links
This presentation is the property of its rightful owner.
1 / 20

Basing Practices on Your Own Evidence: Elevate Your Data PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Danny Singley, Ph.D. Director of Curriculum Development and Research Essential Learning Carol Hurst, Ph.D., LCSW Clinical Quality Enhancement Instructor and Coach Corporate University of Providence Providence Service Corporation. Basing Practices on Your Own Evidence: Elevate Your Data.

Download Presentation

Basing Practices on Your Own Evidence: Elevate Your Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Danny Singley, Ph.D.

Director of Curriculum Development and Research

Essential Learning

Carol Hurst, Ph.D., LCSW

Clinical Quality Enhancement Instructor and CoachCorporate University of ProvidenceProvidence Service Corporation

Basing Practices on Your Own Evidence:

Elevate Your Data

  • Overview

  • Do’s and Don’ts for constructing surveys

  • Setting up custom surveys in Elevate

  • Choosing the report format that’s right for you

  • Results from a recent research project conducted with Providence


  • Uses in organizations

  • Who develops them

  • Data collection methods

  • Disseminating and acting on results

Developing Surveys – Do’s

  • Chefs in the kitchen

  • Keep the outcome in mind

  • Use existing forms and data

  • Items -less is more

  • Meaningful sub-scales

  • Standardize the items and response mode

  • Balanced positive/negative frame

  • Incentivize and follow up

Developing Surveys – Don’ts

  • Too many items – respondent fatigue

  • Messy/Non-uniform item types

  • All qualitative responses

  • “Neutral” responses

  • “Double-barreled” items

  • Negatively-framed items

  • Burying results

  • Going beyond the results

Surveys in Elevate

  • Post-tests vs. surveys

  • Create a “sham course”

  • Test it

  • Create survey curriculum

  • Assign to respondents

  • Sit back and let the data roll in

  • Reports


  • Reports

  • Aggregate data

  • Raw data

  • Intended audience

  • Sound bites and pictures

E-Learning Effectiveness Research

Early Research Findings (1994-2006)

  • E-Learning and face-to-face equally effective

    DoE’s (2009) Meta-Analysis – 51 studies

  • Both pure and blended online learning are superior to face-to-face

  • E-learning enhances learning with more time and reflection exercises

  • Blended training more effective than pure online training when compared with face-to-face

Our Training Effectiveness Study

Addressed how learners 159 Providence clinicians benefited from taking the same five module course -Making Parenting Matter: Coaching Parents on Positive Parenting - in four different conditions:

  • Live Workshop – one day (n = 46)

  • Tele-class - five facilitated weekly conference calls (n = 46)

  • E-learning - five weekly e-learning courses (n = 45)

  • Waitlist group (n = 22)

The MPM Survey

  • Developed for this study

  • Face valid

  • 43 items pre-test

  • 36 items post-test and follow-up

  • Four subscales:

    Applicability, Understanding

    Self-Efficacy, Utilization

Goals and Hypotheses of the Study


  • Evaluate the different modes of training

  • Compare ROI for different modes

  • Incorporate EL Connect into learning

  • Use Elevate for data collection


  • Intervention groups will show improved training outcomes

  • Tele-class and e-learning will show greatest ROI

MPM E-Learning Course: Meet Carol’s Avatar


Social Networking on ELC

Results - Training Effectiveness

Results - ROI


  • Some participants dropped out

  • More attrition in e-learning and tele-class conditions compared to workshop

  • Themes from qualitative feedback:

    - Technical difficulty

    - Job demands

    - Preference for face-to-face


  • E-learning, tele-classes, and face-to-face showed comparable learning outcomes as compared with the control group.

  • E-learning and tele-class

    trainings are considerably

    more cost-effective than

    face-to-face workshops.


Dillman, D.A. (2007). Mail and Internet Surveys: The Tailored Design Method, Wiley: Hoboken, New Jersey.

Sudman, S., Bradburn, N.M, & Schwartz, N. (1996). The Application of Cognitive Processes to Survey Methodology, Jossey-Bass, San Francisco.

U.S. Department of Education, Office of Planning, Evaluation, and Policy Development (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies, Washington, D.C.


  • Take care to design useful surveys

  • Elevate is a key data collection resource

  • Match your reports to your audience

  • E-learning and tele-classes were as effective than face-to-face workshops in terms of learning outcomes

  • E-Learning was the most cost-effective of all the training modalities in this study

Go Forth and Collect Data

Thank you very much!!

For more information, contact:

Dr. Danny Singley

Dr. Carol Hurst

  • Login