Online vs Face to Face : Is There a Significant Difference in Learning Outcomes?

1. Online vs Face to Face: Is There a Significant Difference in Learning Outcomes? Elaine Gerber & AJ Kelton, Montclair State University &Tracy Chu, Brooklyn College, CUNY

2. Abstract Our aim is to better understand whether and how instructional modality (online vs face-to-face) impacts student learning. More specifically, this presentation compares survey results and final grades from an in-person face-to-face class with those from an online class, where the professor, course, and semester are the same. While much has been written about online learning, it is rare for the same course to be taught by the same professor in the same semester at the same institution. Thus, these circumstances provided a unique opportunity for comparison. Consistent with extant research, our preliminary findings suggest that there is no significant difference in learning outcomes, according to modality. What did appear significant, however, is that the online students worked more hours and experienced more life obstacles. Despite working more hours and having more obstacles, the online students performed as well in the course overall, as students who didn’t experience as many challenges or spend as many hours in paid employment. This suggests that certain attributes of online learners, such as being highly organized, a “self-starter,” or previously performing well in school, may be better predictors of success than the modality used to deliver course content. This research is ongoing in order to gather a larger sample with greater predictive power. Our presentation at the RAUL Showcase 2014 will highlight findings from the first round of data collection (Spring 2013).

3. Research Design A “natural experiment” • Same course • Same professor • Same school • Same semester RQ: Is there a significant difference in learning outcomes by modality?

4. Same Course?! Similar / shared the same: • course objectives, readings • enrollment size (35 students each) • Instructor,semester, university Differences: • “lecture” content • nature, amount, and quality of peer-based discussion • presence of the instructor • amount of writing involved • weights attributed to various assignments • type of student taking each modality(below)

5. Other limitations? • How similar is the course really? (see above) • Planning / Intentional course structure for research design vs retro-fitting to work with existing course offerings • Intellectual property issues • Self-reports, and the problems that go with it • Small sample size • Student characteristics / attributes

6. Sample Bias?

7. Data Collection • Subjects were recruited from the full class roster of all students who were enrolled in ANTH 110-01 (face-to-face) and ANTH 110-03 (online) at Montclair State University, Spring 2013 • Students were given 2 points on their final exam for participating; they also had the option to complete a short assignment in lieu of participating, in order to receive the same extra credit.   • The professor did not know which students had completed the survey and which had done the optional assignment, only which students were to receive the extra two points on their final exam, until after grades were submitted to the Registrar.  

8. Data Analysis Data were then exported from the Limesurvey database into SPSS for analysis.  Most questions were close-ended and could therefore be analyzed using this program to measure frequencies and other descriptive statistics.  We used T-tests for correlations and a multiple regression analysis, which I’ll talk more about below. The few open-ended questions were analyzed “by hand,” by reading and re-reading the data in either MS Word or Excel.  There was not sufficient open-ended content to warrant using a qualitative software analysis program, such as The Ethnograph.

9. Preliminary Findings • Still preliminary, ongoing • Collecting “round 2” data Spring 2014 Nonetheless…….. What did we find?

10. Statistical Findings: Only significant difference between the online and F2F samples is that: online students worked more hours, and experienced more obstacles! But, they did just as well in the course overall…

11. Grades by modality

14. Obstacles Are there any obstacles to learning that you faced this semester? (check all that apply): Working too many hours Commute to class and/or parking often caused me to miss class I had family emergencies I had personal issues I had housing problems I was too busy Access to computers & Internet were not reliable Course content too intellectually difficult or presented too poorly to learn Other (please explain):  ______________________________

15. Predictors of Learning Outcome When looking at the entire sample overall (n=41) , the only variables that correlate with FINAL GRADE are RACE and GPA. When looking at each modality separately, in the online class (n=17), RACE was correlated to FINAL GRADE, but GPA was not. However in the F2F class (n=24), GPA was correlated to Final Grade, and RACE was not.

17. When GPA is controlled for, race is no longer a significant predictor of course grade (Model 1 & Model 2). Moreover, modality does NOT predict course grade, even when controlling for race and GPA (Model 3). Controlling for modality and race, only significant predictor of course grade is previous GPA (Model 3).

19. Discussion What is going on here?

20. Other Predictors of Outcome? Do you consider yourself… (select all that apply): very organized highly motivated to learn an independent worker able to multi-task good with time management focused & goal-oriented comfortable with technology other: _________________

21. Implications Not all students are the same, no universal student Implications for academic advising: • Retention • Time to graduation Questions? Suggestions? …And, thanks!