1 / 46

Charles D. Dziuban Patsy D. Moskal University of Central Florida

A Distributed Learning Impact Evaluation: What UCF Knows About Its Online Students, Faculty, Successes, and Challenges. Charles D. Dziuban Patsy D. Moskal University of Central Florida. The University of Central Florida. UCF terminology for courses utilizing web instruction.

morse
Download Presentation

Charles D. Dziuban Patsy D. Moskal University of Central Florida

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Distributed Learning Impact Evaluation:What UCF Knows About Its Online Students, Faculty, Successes, and Challenges Charles D. Dziuban Patsy D. Moskal University of Central Florida

  2. The University of Central Florida

  3. UCF terminology for courses utilizing web instruction • “Web” courses: delivered entirely over the Web, with no regular class meetings • “Mixed-mode” courses: some face-to-face instruction is replaced with web instruction so that on-campus time is reduced • “Enhanced” courses: delivered entirely in face-to-face mode, but with web enhancements

  4. UCF Enrollment by Web modality Enrollment Spring 2000 Fall 2002 Semester

  5. Scope of UCF’s Online Program • Half (600) of all full-time faculty engaged • Half (300) of these trained through IDL6543 • 1,500+ courses in WebCT • 900+ courses built with production support • Four online bachelor’s degree completion programs • Five online master’s degree programs • Four online graduate certificate programs

  6. The Evaluation

  7. Principles that guide our evaluation • Evaluation must be objective. • Evaluation should conform to the culture of the institution. • Uncollected data cannot be analyzed. • Data do not equal information. • Qualitative and quantitative approaches must complement each other. • We must show an institutional impact. • Our results may not be generalized beyond UCF.

  8. Students Success rates Attitudes Demographic inertia Withdrawal rates Strategies for success Reactive behavior patterns Distributed Learning Impact Evaluation components Faculty Online programs Real-time surveys Writing project model Modified instructional theories Student evaluation of instruction Large online classes

  9. Student Results

  10. Student satisfaction in fully online and mixed-mode courses 44% Fully online (N = 1,526) 41% 39% 38% Mixed-mode (N = 485) 11% 9% 9% 5% 3% 1% Very Satisfied Neutral Very Unsatisfied Satisfied Unsatisfied

  11. Students’ likelihood of taking another web course Fully online (N = 1,526) 49% 47% Mixed-mode (N = 485) 29% 23% 16% 11% 10% 8% 6% 2% Definitely Probably Not Sure Probably Not Definitely Not

  12. What students like most about W courses 41% 29% 18% Flexibility Convenience Improved Logistics

  13. What students like least about W courses 16% 16% 11% 11% 5% Reduced or no F2F time Lack of instructor availability Technical problems Excessive requirements Facilitates procrastination

  14. Success rates of modalities excluding computer science -- Spring 00 through Summer 01 F2F Total N= 57,950 students E M W 95 95 94 93 92 91 90 90 89 89 88 89 88 88 86 86 85 84 84 82

  15. An example of a fractured data set Course Type Course Prefix W M E F2F POS2041 * * * EDF6481 * * NUR6465 * * SYG2010 * * *

  16. A segment model for success Overall 85.9% n=11,286 Arts & Sciences, Business Admin., Hospitality Mgmt. Health & Pub. Affairs Engineering Education 85.8% n=6,460 72.7% n=378 91.5% n=2,079 86.7% n=2,369 F2F, E, M W F2F E, M, W F2F E, M 89.1% n=1,043 79.6% n=230 94.1% n=1,036 64.7% n=148 74.8% n=821 86.5% n=5,639 females males A&S BA & Hosp. mgmt 84.1% n=2,376 78.5% n=526 88.4% n=3,263 68.9% n=298

  17. Student Behavior Types

  18. Research on reactive behavior patterns • Theory of William A. Long, University of Mississippi • Ambivalence brings out behavior patterns • Provides a lens for how “types” react to different teaching styles

  19. Resources • Personality • Emotional maturity • Sophistication level • Level of intellect • Educational level • Character development

  20. Aggressive Independent high energy action-oriented not concerned with approval speaks out freely gets into confrontational situations Passive Independent low energy not concerned with approval prefers to work alone resists pressure from authority Aggressive Dependent high energy action-oriented concerned with approval rarely expresses negative feelings performs at or above ability Passive Dependent low energy concerned with approval highly sensitive to the feelings of others very compliant A description of Long behavior types

  21. Phobic exaggerated fears of things often feels anxious often sees the negative side doesn’t take risks Compulsive highly organized neat, methodical worker perfectionist strongly motivated to finish tasks Impulsive explosive quick-tempered acts without thinking frank short attention span Hysteric dramatic and emotional more social than academic artistic or creative tends to overreact A description of Long behavior traits

  22. Distribution of Long types and traits for fully online students 75% PD 7% 51% AD 54% AI 21% 30% 26% PI 18% (N=1,520) (N=1,437)

  23. Distribution of Long types and traits for mixed-mode students 76% PD 8% 54% AI 17% AD 52% 32% 23% PI 23% (N=472)

  24. Distribution of Long types and traits for Composition I students PD 14% 53% 50% 40% 38% AI 20% AD 44% PI 23% (N=1,054)

  25. Long types and traits for Web, mixed-mode, and general education students Types Traits

  26. Faculty Results

  27. Time to develop course as compared with a comparable face-to-face section A lot more time More work A little more time 52% 77% About the same A little less time A lot less time 43% 21% Equal to or less than 5% 2% W n=56 M N=43 Modality

  28. Time in weekly course administration activities as compared with a comparable face-to-face section A lot more time 43% More work A little more time 60% About the same A little less time 38% 20% A lot less time 15% 19% Equal to or less than 2% 4% W n=55 M N=42 Modality

  29. Time in weekly course delivery activities as compared with a comparable face-to-face section A lot more time A little more time About the same A little less time More work 9% 5% A lot less time 15% 13% 28% 37% Equal to or less than 30% 29% 20% 15% W n=55 M N=42 Modality

  30. Amount of interaction in Web classes compared to comparable F2F sections Increased Somewhat increased 45% More interaction 62% About the same Somewhat decreased 30% Decreased 16% Equal to or less than 13% 15% 8% 2% 3% 7% W n=55 M N=40 Modality

  31. Quality of interaction in Web classes compared to comparable F2F sections Increased Somewhat increased About the same 30% 35% Better interaction Somewhat decreased Decreased 37% 33% Equal to or less than 22% 19% 9% 14% 2% W n=55 M N=43 Modality

  32. Faculty satisfaction compared with a comparable face-to-face section 38% Very satisfied 44% 49% Satisfied Positive Neutral 58% Unsatisfied 44% 38% Very unsatisfied Neutral or negative 7% 6% 5% 5% 7% W n=55 M N=43 F2F N=64 Modality

  33. Faculty willingness to teach Web courses in the future Definitely Probably Probably not Definitely not Positive 81% 69% 16% 13% Neutral or negative 2% 10% 4% 6% W n=71 M N=53 Modality

  34. Relationships of faculty satisfaction with class interaction and workload (TAU-b) W M (n=53) (n=38) Amount of interaction .39** .34* Quality of interaction .43** .51** Time to develop .16 .09 Time to administer .10 .01 Time to deliver .06 .10 *p<.05; ** p<.01

  35. What faculty like most about teaching on the web 26% 26% 19% 11% Convenience of location Flexibility of time Interaction with students Enjoy technology

  36. What faculty like least about teaching on the web 35% 18% 14% Lack of face-to-face contact Technical problems Requires too much time

  37. Student Ratings

  38. Student Evaluation of Instruction • Same form used for all classes -- F2F, and Web • Some concern among Web faculty that “standard” form not appropriate for asynchronous courses • Concern among faculty that Web faculty received lower evaluations than F2F

  39. Student Ratings by Modality Very Modality Excellent Good Good Fair Poor F2F 42.00 29.50 19.00 7.20 2.40 (N=628,623) E 44.00 29.10 17.40 6.90 2.60 (N=6,632) M 40.60 28.60 20.60 7.70 2.40 (N=11,450) W 55.40 25.20 12.10 4.90 2.50 (N=5,435)

  40. Research Initiative for Teaching Effectiveness For more information contact: Dr. Chuck Dziuban (407) 823-5478 dziuban@mail.ucf.edu Dr. Patsy Moskal (407) 823-0283 pdmoskal@mail.ucf.edu http://pegasus.cc.ucf.edu/~rite

  41. Rationale for the Study • Teaching evaluation data for a 3-year period were available to allow a comparison of two different sets of items (UCF/BOR).  • Responding to faculty interest, the UCF Faculty Senate requested that an evaluation of the Student Evaluation of Instruction measures be performed. (FS 1995-96-11)

  42. Approximately 450,000 student responses Five Colleges: Arts and Sciences Business Administration Education Engineering Health and Public Affairs The Study Layout • Three Levels: • Lower Undergraduate • Upper Undergraduate • Graduate • Three Years: • 1996-97 • 1997-98 • 1998-99

  43. Rules Leading to Overall Rating of Excellent N=46,805 Probability = .96

  44. Rules Leading to Overall Rating of Excellent N=3,462 Probability = .85

  45. Rules Leading to Overall Rating of Excellent N=6,215 Probability = .78

  46. Overall Excellent Ratings by College:Unadjusted and Adjusted for Rules

More Related