1 / 43

Donald Staub, Title III Director Carteret Community College

“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online. Exploring the Assessment of Distance Learning.

reia
Download Presentation

Donald Staub, Title III Director Carteret Community College

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online

  2. Exploring the Assessment of Distance Learning ? ? ? ? ? ? Donald Staub, Title III Director Carteret Community College

  3. The Lineup • DL @ CCC • Why Worry about DL Assessment? • How We’re Doing It • The DL Program • DL in Programs • Barriers, Conclusions & Solutions ccctitle3.wordpress.com

  4. DL @ CCC • 34 programs @ CCC • SP10 enrollment ~1800 • DL is 10+ years old • Spring 2010 • 386 total sections taught • 193 seated sections (50%) • 75 Internet (20%) • 41 Hybrid (10%) • 77 Web-Based (20%)

  5. DL @ CCC 248 seated sections 62 internet sections (18 FT / 19 PT) 22 hybrid sections (13 FT / 9 PT) 37 web-based sections (25 FT / 9 PT) Fall 2009 data

  6. Why worry about DL assessment? College-Level: Reason #1: Student Success Reason #2: Persistence (btw: success vs. retention vs. persistence)

  7. Course-Level: Reason #3: Generalizability (c.f. Oosterhof, et al, 2008) “Generalizability is the degree to which learner performances that are observed generalize to performances that are not observed.”

  8. Reason #4 SACS

  9. “Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” • (CS 3.3.1) • COC Distance Education Policy Statement • http://www.sacscoc.org/pdf/081705/distance education.pdf

  10. (Jan 2010) http://www.sacscoc.org/policies.asp

  11. Q1: Is the distance education program reflected in the institution’s strategic (or master) plan? • 2009-2012 Strategic Plan • We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion. PM 16—Success and Withdrawal Rates by Instructional modalities

  12. Q2: Is there evidence that outcomes for the program have been identified? AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.

  13. Q2: Is there evidence that outcomes for the program have been identified? Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. Assessment: Quality Assessment Plan, Success/Persistence Rates, Student Surveys/Evaluations. AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC. Assessment: The number of internet/hybrid/web-based sections offered.

  14. The QAP(Quality Assessment Plan) • Developed in-house (using models) • Peer-reviewed by content specialist • Stipends (T3) • All current online courses have been reviewed • All new online courses must be reviewed/ certified • http://web.carteret.edu/keoughp/TitleIII/FinalQAP.xls

  15. The QAP Score Card

  16. The QAP Score Card: Learning

  17. Success - Fall 2009 • 248 seated sections • 72% success / 25% D, F, WD (14% W) • 62 internet sections • 68% success / 31% D, F, WD (16% W) • 22 hybrid sections • 75% success / 24% D, F, WD (9% W) • FA08 internet: 61 sections; 67% success • FA07 internet: 59 sections; 66% success • FA06 internet: 48 sections; 66% success

  18. Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? • Modality Comparison • Grade Distributions • ILLOs (Gen Ed outcomes) • Program Review • Evaluate Instructional Modalities

  19. Modality Comparison institutional

  20. Modality Comparison Gateway courses

  21. Modality Comparison Program-level

  22. Grade Distributions Fall 2008

  23. ILLOs (Gen Ed outcomes) • Communication • Computer Literacy • Critical Thinking • Humanities & Fine Arts • Information Literacy • Personal Growth & Responsibility

  24. Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment:Final Exam in CIS 110. This exam consists of a hands-on skills assignment for Word, Excel, Access and PowerPoint and a 25 question Multiple choice assignment with questions regarding computer hardware, system software, internet browsers and other related computer terminology.

  25. Computer Literacy - CIS 110 Same instructor teaching one section of each

  26. Personal Growth & Responsibility ACA 115 Outcome: Students demonstrate the ability to understand and manage self effectively. Assessment: Advisement assignment in ACA 115 & 118 evaluated by rubric. Results Spring 2009: 131 total projects submitted; 81% scored 80 or above. Seated 115: 85% scored 80 or above Online 115: 81% scored 80 or above

  27. Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by “traditional” methodologies, is assessment identified for distance learning students separate from students taking courses by “traditional” methodologies? • Yes! … No?…maybe?? • PLLOs • Course-Level Assessment

  28. PLLOs (SLOs) Same instructor teaching one section of each (take a look at a common assignment from each section) • ACC 120 • BUS 110 • ECO 251/252

  29. ACC 120

  30. BUS 110 ECO 250

  31. Course-level assessment • ART 111 • HIS 111 • RCP 114

  32. ART 111

  33. A - (100 - 90): - You posted 3 or more times and you completed the Assignment with a well thought through response. (The more you post MEANINGFUL responses over 3 the higher your grade. Though welcome and expected, please understand that a One or Two sentence "feel good" post such as (... Great response Michaela!!) or (... I love how you put the meaning and then went down and talked about them. it made it easy to understand!!) will count for very little towards your grade.

  34. HIS 111 Essays (online) vs. Research Paper (seated) … same rubric

  35. RCP 114

  36. Conclusions “I got rid of [the discussion board]…it’s a pain in the ass.” “I have to pull teeth to get them to post even when they get a score” 20-30% of entire grade is Db…“but it’s not really a discussion board… it’s more of an assignment tool”

  37. “Participation is reserved for those on the bubble.” “There’s nothing like a good old test. It averages. Students can’t argue the grade. They’re either right or wrong.”

  38. Donald Staub Title III DirectorCarteret Community Collegestaubd@carteret.edu 252- 222-6010 http://ccctitle3.wordpress.com/

More Related