Defining and Implementing Quality Assurance Standards for Online Courses Lawrence C. Ragan, Director, Instructional Design/Development, The Pennsylvania State University Christina M. Sax, Professor & Assistant Dean, Social, Behavioral, Natural, & Mathematical Sciences, University of Maryland University College
Presentation Agenda • Session Objectives • Participant Activity • QM Overview: Circle and Rubric • Penn State Approach/Status • Story of Convergence • QM Attractive Features • PSU Adoption/Adaptation • QM Opportunities • a. Models • b. Institution/Institute
Session Objective • Provide an overview of the development, implementation and administration of an inter-institutional quality assurance system and the application within a single institution.
Participant Activity • 2-minute drill • Turn to person next to you and discuss: • What characteristics would you use to assess the quality of the vehicle you may be purchasing? • Feedback from Teams • Issues of Definition--single vs group
Quality Matters • Quality does matter to … • students • faculty • administrators • institutions • consortia • accrediting agencies • legislators • tax-payers How do we … • identify & recognize it? • motivate & instill it? • assess & measure it? • insure it? • assure it?
Quality Matters: Inter-Institutional Quality Assurance in Online Learning • Grantor: FIPSE • Grant period: 9/03 – 8/06 • Award: $509,177 • Grantee: MarylandOnline • Statewide consortium: 14 community colleges, 5 senior institutions • http://www.QualityMatters.org
FIPSE Interested Because … • Quality assurance of online courses is important • Voluntary inter-institutional assurance has never been done before • This can serve as a national model Quality Matters!
Course Meets Quality Expectations Course Revision Incoming process • Institutions • CAO’s • AR’s Faculty Course Developers National Standards & Research Literature Course Rubric Faculty Reviewers Training Peer Course Review Feedback Instructional Designers
More than average; more than “good enough” An attempt to capture what’s expected in an effective online course at about an 85% level Based on research and widely accepted standards For Our Purposes, Quality Is… 85 %
Major Themes • develop inter-institutional consensus - criteria & process online course QA • assure & improve course quality • positively impact student learning • faculty-centered activities • promote voluntary participation and adoption • ensure institutional autonomy • replicable, reliable, and scalable processes • foster sharing of materials and expertise • create opportunities for training and professional development
What Quality Matters is NOT • Not about an individual instructor (it’s about the course design) • Not about faculty evaluation (it’s about course quality) • Not a win/lose, pass/fail test (it’s about a continuous improvement process in a supportive environment)
Quality Matters Rubric • Based on • research literature • nationally recognized standards of best practice • instructional design principles • Used by review teams to: • assess course quality in 8 key areas (40 review elements) • provide feedback to faculty course developer • provide guidance to instructional design support team
Rubric Scoring • Team of three reviewers • One score per standard based on majority • Two criteria to meet quality expectations: • “Yes” to all 14 Essential Standards • Receive at least a total of 68 points
Review Teams • Teams composed of 3 reviewers: • 1 from home institution, 2 from others • 1 from same discipline, 2 from others • May be either faculty and or ID/IT (practitioners) • mix of CC & 4 yr schools • mix of large & small schools • mix of public & private schools • Course author--resource for review team
Reviewer Rubric Training • Focus on: • Application of rubric to course review • Interpretation of review elements • Providing constructive feedback • Competency-based
Course Reviews • To date, 50% meet expectations on initial review • Instructional design support provided • Identified 11 common areas for improvement • Target for course development/revision, faculty training • instructor's self-introduction • netiquette expectations • learning objectives stated at the module/unit level • self-check/practice activities with feedback • Interaction • links to school's academic support • ADA issues
What’s In It For Institutions … • External validation process • Strengthens institution’s accreditation package • Raise QA as a priority activity • Gain access to a sustainable, replicable, scalable QA process • Inform online course training & practices • Provide professional development activities
What’s In It For Participants … • improve your online course • instructional design support • external quality assurance • expand professional community • chance to review other courses • gain new ideas for your own course • participation useful for annual evaluations, promotion applications, professional development plan/requirements
National Participation • Scope: • individuals from 70 different institutions (including the 19 MOL schools) in 14 different states • Over 250 faculty trained to review online courses using the rubric • National external partners & advisory board • Use of QM System: • online course development, review, and revision, faculty training, formation of distance learning policies & steering committees, institutional reaccredidation packages
Awards - 2005 • WCET Outstanding Work (WOW) Award • USDLA 21st Century Best Practice Award • Maryland Distance Learning Association (MDLA) Best Program Award
PSU Approach to Quality Standards • History: WC started in 1998 with 4 courses grew to 150 by 2002 • Course design and development initiatives from units external to WC • “Some” variability of design and development practices • Compromise of “PSU Quality” • If delivered via WC then held responsible for ensuring quality
Definition of “Quality” The user (learner) of a system (educational) has a reasonable opportunity for success.
PSU Online Quality Standards • Charge by Provost to eLearning Council to identify standards of quality for online learning • Built upon technical standards and pedagogical guidelines developed by WC • Forwarded to Provost and accepted as University policy by the University Web Task Force
Implementation • Distributed to Deans as policy (probably stopped there) • Do exist as record somewhere and accessible if you can find them • No enforcement “teeth”-- no implementation system established • No mechanism for reviewing and updating • Some may claim “academic freedom” and standards need not apply
PSU Interest in QM Standards • PSU eLearning Council examining the potential of QM model at PSU • Subcommittee organized to look at various adaptation options • Primary points of interest: • Externally validated • Well defined rubric • Established “system” • Research-based criteria
Potential Options • PSU can fully accept and adopt the QM system for quality assurance. • Courses would be submitted to QM for evaluation and ultimately the QM seal of approval. • PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process. • PSU would negotiate with the QM board on the adaptation process in order to maintain the QM seal of approval. • PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process. • PSU would not seek the QM seal of approval. • Penn State would maintain the current model of Technical Standards and Pedagogical Guidelines.
QM: Looking Ahead • Adapt rubric & process for other formats • Hybrid/blended, face-to-face, continuing education, commercial, professional training • Adapt rubric & process for specific institutional needs • Explore the “QM Institution” concept • Assess the impact on student learning through research projects
QM: Looking Ahead • Diversify Training Program • Additional review training & train-the-trainer • Annual rubric update cycle • Sustainability plan • Develop partnerships & business opportunities
www.qualitymatters.org For more information: Kay Kane Kkane@pgcc.edu