defining and implementing quality assurance standards for online courses l.
Skip this Video
Loading SlideShow in 5 Seconds..
Defining and Implementing Quality Assurance Standards for Online Courses PowerPoint Presentation
Download Presentation
Defining and Implementing Quality Assurance Standards for Online Courses

Loading in 2 Seconds...

play fullscreen
1 / 29

Defining and Implementing Quality Assurance Standards for Online Courses - PowerPoint PPT Presentation

  • Uploaded on

Defining and Implementing Quality Assurance Standards for Online Courses Lawrence C. Ragan , Director, Instructional Design/Development, The Pennsylvania State University

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Defining and Implementing Quality Assurance Standards for Online Courses

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
defining and implementing quality assurance standards for online courses

Defining and Implementing Quality Assurance Standards for Online Courses

Lawrence C. Ragan, Director, Instructional Design/Development, The Pennsylvania State University

Christina M. Sax, Professor & Assistant Dean, Social, Behavioral, Natural, & Mathematical Sciences, University of Maryland University College

presentation agenda
Presentation Agenda
  • Session Objectives
  • Participant Activity
  • QM Overview: Circle and Rubric
  • Penn State Approach/Status
  • Story of Convergence
  • QM Attractive Features
  • PSU Adoption/Adaptation
  • QM Opportunities
  • a. Models
  • b. Institution/Institute
session objective
Session Objective
  • Provide an overview of the development, implementation and administration of an inter-institutional quality assurance system and the application within a single institution.
participant activity
Participant Activity
  • 2-minute drill
    • Turn to person next to you and discuss:
    • What characteristics would you use to assess the quality of the vehicle you may be purchasing?
  • Feedback from Teams
  • Issues of Definition--single vs group
quality matters
Quality Matters
  • Quality does matter to …
  • students
  • faculty
  • administrators
  • institutions
  • consortia
  • accrediting agencies
  • legislators
  • tax-payers

How do we …

  • identify & recognize it?
  • motivate & instill it?
  • assess & measure it?
  • insure it?
  • assure it?
quality matters inter institutional quality assurance in online learning
Quality Matters: Inter-Institutional Quality Assurance in Online Learning
  • Grantor: FIPSE
    • Grant period: 9/03 – 8/06
    • Award: $509,177
  • Grantee: MarylandOnline
    • Statewide consortium: 14 community colleges, 5 senior institutions
fipse interested because
FIPSE Interested Because …
  • Quality assurance of online courses is important
  • Voluntary inter-institutional assurance has never been done before
  • This can serve as a national model

Quality Matters!


Course Meets Quality Expectations

Course Revision

Incoming process

  • Institutions
    • CAO’s
    • AR’s

Faculty Course Developers

National Standards & Research Literature



Faculty Reviewers


Peer Course Review


Instructional Designers

for our purposes quality is
More than average; more than “good enough”

An attempt to capture what’s expected in an effective online course at about an 85% level

Based on research and widely accepted standards

For Our Purposes, Quality Is…

85 %

major themes
Major Themes
  • develop inter-institutional consensus - criteria & process online course QA
  • assure & improve course quality
  • positively impact student learning
  • faculty-centered activities
  • promote voluntary participation and adoption
  • ensure institutional autonomy
  • replicable, reliable, and scalable processes
  • foster sharing of materials and expertise
  • create opportunities for training and professional development
what quality matters is not
What Quality Matters is NOT
  • Not about an individual instructor

(it’s about the course design)

  • Not about faculty evaluation

(it’s about course quality)

  • Not a win/lose, pass/fail test

(it’s about a continuous improvement process in a supportive environment)

quality matters rubric
Quality Matters Rubric
  • Based on
    • research literature
    • nationally recognized standards of best practice
    • instructional design principles
  • Used by review teams to:
    • assess course quality in 8 key areas (40 review elements)
    • provide feedback to faculty course developer
    • provide guidance to instructional design support team
rubric scoring
Rubric Scoring
  • Team of three reviewers
  • One score per standard based on majority
  • Two criteria to meet quality expectations:
    • “Yes” to all 14 Essential Standards
    • Receive at least a total of 68 points
review teams
Review Teams
  • Teams composed of 3 reviewers:
    • 1 from home institution, 2 from others
    • 1 from same discipline, 2 from others
    • May be either faculty and or ID/IT (practitioners)
    • mix of CC & 4 yr schools
    • mix of large & small schools
    • mix of public & private schools
  • Course author--resource for review team
reviewer rubric training
Reviewer Rubric Training
  • Focus on:
    • Application of rubric to course review
    • Interpretation of review elements
    • Providing constructive feedback
    • Competency-based
course reviews
Course Reviews
  • To date, 50% meet expectations on initial review
    • Instructional design support provided
  • Identified 11 common areas for improvement
    • Target for course development/revision, faculty training
      • instructor's self-introduction
      • netiquette expectations
      • learning objectives stated at the module/unit level
      • self-check/practice activities with feedback
      • Interaction
      • links to school's academic support
      • ADA issues
what s in it for institutions
What’s In It For Institutions …
  • External validation process
  • Strengthens institution’s accreditation package
  • Raise QA as a priority activity
  • Gain access to a sustainable, replicable, scalable QA process
  • Inform online course training & practices
  • Provide professional development activities
what s in it for participants
What’s In It For Participants …
  • improve your online course
  • instructional design support
  • external quality assurance
  • expand professional community
  • chance to review other courses
  • gain new ideas for your own course
  • participation useful for annual evaluations, promotion applications, professional development plan/requirements
national participation
National Participation
  • Scope:
    • individuals from 70 different institutions (including the 19 MOL schools) in 14 different states
    • Over 250 faculty trained to review online courses using the rubric
    • National external partners & advisory board
  • Use of QM System:
    • online course development, review, and revision, faculty training, formation of distance learning policies & steering committees, institutional reaccredidation packages
awards 2005
Awards - 2005
  • WCET Outstanding Work (WOW) Award
  • USDLA 21st Century Best Practice Award
  • Maryland Distance Learning Association (MDLA) Best Program Award
psu approach to quality standards
PSU Approach to Quality Standards
  • History: WC started in 1998 with 4 courses grew to 150 by 2002
  • Course design and development initiatives from units external to WC
  • “Some” variability of design and development practices
  • Compromise of “PSU Quality”
  • If delivered via WC then held responsible for ensuring quality
definition of quality

Definition of “Quality”

The user (learner) of a system (educational) has a reasonable opportunity for success.

psu online quality standards
PSU Online Quality Standards
  • Charge by Provost to eLearning Council to identify standards of quality for online learning
  • Built upon technical standards and pedagogical guidelines developed by WC
  • Forwarded to Provost and accepted as University policy by the University Web Task Force
  • Distributed to Deans as policy (probably stopped there)
  • Do exist as record somewhere and accessible if you can find them
  • No enforcement “teeth”-- no implementation system established
  • No mechanism for reviewing and updating
  • Some may claim “academic freedom” and standards need not apply
psu interest in qm standards
PSU Interest in QM Standards
  • PSU eLearning Council examining the potential of QM model at PSU
  • Subcommittee organized to look at various adaptation options
  • Primary points of interest:
    • Externally validated
    • Well defined rubric
    • Established “system”
    • Research-based criteria
potential options
Potential Options
  • PSU can fully accept and adopt the QM system for quality assurance.

• Courses would be submitted to QM for evaluation and ultimately the QM seal of approval.

  • PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process.

• PSU would negotiate with the QM board on the adaptation process in order to maintain the QM seal of approval.

  • PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process.

• PSU would not seek the QM seal of approval.

  • Penn State would maintain the current model of Technical Standards and Pedagogical Guidelines.
qm looking ahead
QM: Looking Ahead
  • Adapt rubric & process for other formats
    • Hybrid/blended, face-to-face, continuing education, commercial, professional training
  • Adapt rubric & process for specific institutional needs
  • Explore the “QM Institution” concept
  • Assess the impact on student learning through research projects
qm looking ahead28
QM: Looking Ahead
  • Diversify Training Program
    • Additional review training & train-the-trainer
  • Annual rubric update cycle
  • Sustainability plan
  • Develop partnerships & business opportunities
www qualitymatters org

For more information:

Kay Kane