“When learners are at a distance, the careful design of assessments is particularly important, bec...
Download
1 / 61

Donald Staub, Title III Director Carteret Community College - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Donald Staub, Title III Director Carteret Community College' - suzy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”

Oosterhof, et al. (2008) Assessing Learners Online


Assessing Distance Learning: assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”Are they learning what we think we’re teaching?

?

?

?

?

?

?

Donald Staub, Title III Director Carteret Community College


http://ccctitle3.wordpress.com assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”/


The lineup
The Lineup assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”

  • DL @ CCC

  • Why Worry about DL Assessment?

  • How We’re Doing It

  • The DL Program

  • DL in Programs & Courses

  • Possibilities & Challenges


Dl @ ccc
DL @ CCC assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”

  • 33 programs @ CCC

  • Enrollment ~1800

  • DL is 10+ years old

  • Spring 2011

  • 399 total sections

  • 261 seated sections

  • 90 Internet (23%)

  • 40 Hybrid (10%)

  • 8 Web-based

5


Why worry about assessing dl
Why worry about assessing DL? assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”

@ the College & Program-Level:

  • Student Success

  • Persistence

  • (btw: success vs. retention vs. persistence)

  • “Good assessment guides good PD/ID”


@ the Course-Level: assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”

Generalizability

(c.f. Oosterhof, et al, 2008)

“…the degree to which learner performances that are observed generalize to performances that are not observed.”

[i.e. are they learning what we think we’re teaching?]


So, how are we doing this? assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”Caveat: We’re not saying this is right or wrong…it’s just the way that we’re currently approaching it.


“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”

(CS 3.3.1)

COC Distance Education Policy Statement

http://www.sacscoc.org/pdf/Distance%20and%20correspondence%20policy%20final.pdf


The former guidelines
The (former) Guidelines campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”


Q1: Is the distance education program reflected in the institution’s strategic (or master) plan?

2009-2012 Strategic Plan

We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion.

PM 16—Success and Withdrawal Rates by Instructional modalities


Q2: Is there evidence that outcomes for the program have been identified?

DL-AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner.

DL-AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.


Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program?

AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner.

Assessment: Quality Assessment Plan, Success/ Persistence Rates, Student Surveys/Evaluations.

AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.

Assessment: The number of internet/hybrid sections offered.


QAP education program is regularly assessed and steps taken for improvement of the program?


The qap quality assessment plan
The QAP education program is regularly assessed and steps taken for improvement of the program?(Quality Assessment Plan)

  • Developed in-house (using models)

  • Peer-reviewed by content specialist

  • Stipends (T3)

  • All current online courses have been reviewed

  • All new online courses must be reviewed/ certified

  • http://web.carteret.edu/keoughp/TitleIII/FinalQAP.xls


The qap score card
The QAP Score Card education program is regularly assessed and steps taken for improvement of the program?


The qap score card learning
The QAP Score Card: education program is regularly assessed and steps taken for improvement of the program? Learning


Qap success persistence
QAP education program is regularly assessed and steps taken for improvement of the program?Success/Persistence


  • Success - Spring 2010 education program is regularly assessed and steps taken for improvement of the program?

  • 75 sections

  • seated: 75% success (persist: 87.1%)

  • internet: 66% success (persist: 81.7%)

  • hybrid: 73% success (persist: 87.8%)

  • FA09 internet: 62 sections; 68% success

  • FA08 internet: 61 sections; 67% success

  • FA07 internet: 59 sections; 66% success

  • FA06 internet: 48 sections; 66% success

20


Qap success persistence program course evaluations
QAP education program is regularly assessed and steps taken for improvement of the program?Success/PersistenceProgram/Course Evaluations


Surveys evaluations dl program
Surveys & Evaluations: education program is regularly assessed and steps taken for improvement of the program? DL Program


Surveys evaluations dl program1
Surveys & Evaluations: education program is regularly assessed and steps taken for improvement of the program? DL Program

23


Surveys evaluations course level
Surveys & Evaluations: education program is regularly assessed and steps taken for improvement of the program? Course-Level


Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program?

  • Modality Comparison

  • Grade Distributions

  • Program Reviews

  • ILLOs (Gen Ed outcomes)


Modality comparison
Modality Comparison education program is regularly assessed and steps taken for improvement of the program?


Modality Comparison education program is regularly assessed and steps taken for improvement of the program?

Institutional-level


Modality Comparison education program is regularly assessed and steps taken for improvement of the program?

Program-level


Modality Comparison education program is regularly assessed and steps taken for improvement of the program?

Course-level


Modality comparison grade distributions
Modality Comparison education program is regularly assessed and steps taken for improvement of the program?Grade Distributions


Grade Distributions education program is regularly assessed and steps taken for improvement of the program? institution-level

%s


Grade Distributions education program is regularly assessed and steps taken for improvement of the program? course-level


Modality comparison grade distributions program reviews
Modality Comparison education program is regularly assessed and steps taken for improvement of the program?Grade DistributionsProgram Reviews


Evaluate instructional modalities
Evaluate Instructional Modalities education program is regularly assessed and steps taken for improvement of the program?


Modality comparison grade distributions program reviews illos
Modality Comparison education program is regularly assessed and steps taken for improvement of the program?Grade DistributionsProgram ReviewsILLOs


ILLOs education program is regularly assessed and steps taken for improvement of the program?(Gen Ed outcomes)

  • Communication

  • Computer Literacy

  • Critical Thinking

  • Humanities & Fine Arts

  • Information Literacy

  • Personal Growth & Responsibility


Computer Literacy - CIS 110 education program is regularly assessed and steps taken for improvement of the program?

Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers.

Assessment: Final Exam in CIS 110.


Computer Literacy - CIS 110 education program is regularly assessed and steps taken for improvement of the program?Section analysis


Computer Literacy - CIS 110 education program is regularly assessed and steps taken for improvement of the program?

Same instructor teaching one section of each


Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by “traditional” methodologies, is assessment identified for distance learning students separate from students taking courses by “traditional” methodologies?


“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”

So, are they learning what we think we’re teaching?


And the answer is… campus-based programs

  • Yes! … No?…maybe??

  • PLLOs

  • CLLOs


Pllos
PLLOs campus-based programs


PLLOs campus-based programs (SLOs)

Same instructor teaching one section of each (common assignment from each section)

  • Business Administration

  • ACC 120

  • BUS 110

  • ECO 251/252


PLLO: campus-based programs preparation of financial statements as part of mid-termAssessed in: ACC 120

45


BUS 110 campus-based programs

ECO 250


Pllos cllos
PLLOs campus-based programs CLLOs


Course-level assessment - CLLOs campus-based programs

  • ACC 120

  • ART 111

  • EDU 119

  • ENG 111

  • HIS 111

  • PSY 150 & 241

  • RCP 114

  • SOC 210

48


Course-level assessment campus-based programs ~ CLLOs

BUS 110 … Students will be able to:

A: Explain Business Ethics and Social Responsibility

B: Describe the major concepts of business, such as Entrepreneurship, Management, Marketing, Human Resource Management, and Financing

C: Describe Business Globalization


Use of Results: campus-based programs “The on-line section needs some extra material to ‘reinforce’ or explain globalization.”


“Students placed more value on the CMS as an effective learning tool when they perceived the CMS as supporting interactivity among student, instructor, peers, content.” (Chou, Peng, Chang, 2010)


ART 111 learning tool when they perceived the CMS as supporting


A - (100 - 90): learning tool when they perceived the CMS as supporting - You posted 3 or more times and you completed the Assignment with a well thought through response. (The more you post MEANINGFUL responses over 3 the higher your grade. Though welcome and expected, please understand that a One or Two sentence "feel good" post such as (... Great response Michaela!!) or (... I love how you put the meaning and then went down and talked about them. it made it easy to understand!!) will count for very little towards your grade.

54


HIS 111 learning tool when they perceived the CMS as supporting

Essays (online) vs. Research Paper (seated) … same rubric


RCP 114 learning tool when they perceived the CMS as supporting


Other possibilities
Other Possibilities learning tool when they perceived the CMS as supporting

  • Downstream assessment

  • E-ports

57


Challenges
Challenges learning tool when they perceived the CMS as supporting


“I got rid of [the discussion board]…it’s a pain in the ass.”

“I have to pull teeth to get them to post even when they get a score”

“Value of discussion board is suspect at best.”


“I don’t know” the ass.”[what texts and assessments the other instructors are using in this multi-section course]

“My first goal is to get the instructors to agree on common CLLOs. Then we’ll work on common assessments.”

Outcome-Assessment alignment


Donald Staub the ass.”

Title III DirectorCarteret Community [email protected]

252- 222-6010

http://ccctitle3.wordpress.com

www.carteret.edu


ad