We Teach … But are they Learning?. Assessing Student Learning in D.L. Lab Science Courses Peter Jeschofnig, Ph.D. Colorado Mountain College 12 th Sloan-C International Conference Nov 9, 2006. Atlantic Monthly : Nov 2005 . "What Does College Teach?
We Teach … But are they Learning?
Assessing Student Learning in D.L. Lab Science Courses
Peter Jeschofnig, Ph.D. Colorado Mountain College
12th Sloan-C International Conference Nov 9, 2006
Atlantic Monthly: Nov 2005
"What Does College Teach?
It's time to put an end to 'faith-based' acceptance of higher education's quality“
former president of Hobart & William Smith Colleges
and Trinity College
Co-director of the "Collegiate Learning Assessment"
Hersh & Merrow – 2005
Declining by Degrees: Higher Education at Risk
All institutions have some form of assessment, often linked to accreditation
Of 1,393 public and private institutions surveyed in 1999, 82% listed “Excellence in Undergraduate Education” as part of mission statement.
However: Direct measures of student learning remain rare!
Chairman Charles Miller, believes that colleges must better measure the skills and knowledge they impart to students, and openly share that information with the public.
“We need to assure that the American public understand through access to sufficient information, particularly in the area of student learning, what they are getting for their investment in a college education.”
"No College Left Behind?"
Inside Higher Ed,
February 15, 2006
Feb. 3, 2006 meeting in San Diego:
Higher education institutions must be more “transparent” in collecting and giving the public useful information about their activities and their performance.
Topics about which colleges should provide more and better information, the various panels suggested, are on their costs and prices, how their graduates fare in the employment market, and their success in imparting knowledge and, more importantly, skills like critical thinking, to their students.
"I'm disappointed that NASULGC (National Association of State Universities and Land-Grant Colleges) seems to be interested in describing ‘the skills and knowledge that students bring to college’ but evidently not the ones they leave with.
While it's considering publishing ‘data on graduation rates, admissions, applications, student demographics and faculty demographics,’ THE OBVIOUS OMISSION OF EVIDENCE OF STUDENT LEARNING SUGGESTS THAT THEY HAVEN'T HEARD THE CLEAR MESSAGE THAT'S BEEN COMING FROM THE STATES AND THE FEDS FOR ABOUT 20 YEARS: THAT THEY WANT TO KNOW WHAT OUR GRADUATES KNOW AND CAN DO.“
Margaret (Peg) Miller
2006 Director of NFCLL
Government mandated assessments have many negative ramifications that should be avoided.
However, that should not deter academic professionals from designing valid and reliable assessment of learning.
A pro-active academic approach to assessment is surely better than a government mandated one!
“The Learning College and its learning facilitators succeed only when improved and expanded learning can be documented for its learners.”
The Primary Questions For
Every Learning College Action:
or expand learning?
... is the systematic, on-going, iterative process of monitoring learning in order to determine what we are doing well and what we must improve.
… and we want students, employers, peers, policy makers, and the public to know …
how well students are able to use the complex knowledge and abilities
articulated as important to their learning.
Ongoing assessment used to modify instruction and improve learning
End of class or program assessment to verify that learning objectives have been met. This is compared to other classes and institutions to gage the effectiveness of courses and instructional programs.
Online enrollments continue to grow at rates faster than for the overall student body…
Schools expect the rate of growth to further increase and believe that online learning is critical to their long term strategy…
Three quarters of academic leaders at public colleges and universities believe that online learning qualityis equal to or superior to face-to-face instruction. And they expect online offerings to continue to get better.
The Sloan Consortium, Nov. 2004
Entering the Mainstream
The Quality and Extent of Online Education
in the United States, 2003 and 2004
Using Nationally Normed Exams:
Institution Designed Exams:
If we are to avoid seeing a continuing decline in science literacy in America,
lab sciences MUST be fully included in the increasing mix of online course offerings.
However, there are still many instructors and institutions that do not believe lab sciences can effectively be taught at a distance. Valid and reliable assessment data is required to dispel this misconception.
There is ample anecdotal evidence of student learning and satisfaction in distance science courses that use home-based lab kits to fulfill laboratory requirements.
However, there has been little quantitative data to support this positive conclusion.
To quantitatively assess and compare the performance of my chemistry students
In a face-to-face (F2F) chemistry course
with an on-campus laboratory
In an online chemistry (DL) course
with a home-based laboratory kit
Administer and Compare Results for Campus-Based CHE-111 Students and Online CHE-111 Students:
1. American Chemical Society Standardized Exam
Pre-test at the beginning of the semester and
Post-test at the end of the course
2. Traditional homework, quiz, and exam grades
3. Laboratory reports graded via a specific rubric
Scores based on 3007 students in 20 colleges and universities, including:
University of Alabama Albuquerque Technical College
Mercer University Kennesaw State University
Jamestown CC Florida Southern University
Miami University Monroe CC;
University Pittsburgh University of N.Carolina
Title Page - 5 points
Succinct and descriptive title and experiment number, author’s name, partners’ names, course name and number; date of experiment, date of lab report.
Abstract – 10 points
A brief one to two paragraph statement of the purpose of the experiment and the results (i.e. relative yield, identification of unknown, etc). 10 points
Purpose/Hypothesis - 10 points
A detailed statement about the experiment’s purpose and hypothesis (your predictions). Describe what you think the likely outcome of the experiment will be; what scientific principle or law will be tested; what scientific relationship will be shown. Hypothesis should include any relationship between variables and should mention the independent vs. dependent variable.
Procedure – 10 points:
If you followed a detailed procedure from the lab manual, a very short procedure summary will be enough. If you used a procedure not detailed in the lab report, you need to write the procedure in enough detail, using numbered steps, so that someone should be able to repeat your procedure.
This section should include any alterations or errors you made to your operating methodology. Be thorough and specific! Another chemist should be able to read your procedure and reproduce your experiment with precision. This section should not contain any results/data from the exercise. 10 points
Data/Observations - 25 points
A detailed presentation of all hard data gathered during the experiment, as well as an organized presentation of all your observations during the lab. Describe not only what you see, but what it signifies in chemical terms. What is really happening? Also, answer any questions the lab poses here.
Tables should be used whenever possible. Graphs need to be complete with titles, axes must be labeled; the independent variable should be on the x-axis, the dependent variable on the y-axis. Any calculations should be in this section as well. They should be presented clearly and explained. The source of all numbers used in calculations should be included.
Results/Analysis - 20points
A comprehensive, thoughtful discussion of what your data means and how it proved or disproved your hypothesis, including the relationship of the variables as presented by the data. Explain any trends in the data that will be used. You should present the evidence that will support your conclusions. Error analysis, including percent error, should be included here. Are the results consistent within the limitations of equipment and random error. All questions should be answered here.
Conclusion -20 points
A detailed discussion of how the lab results compare with your predictions, and what the results mean in a practical, real world sense. How can your discoveries be applied? Have you verified your hypothesis? Did you demonstrate the scientific principle of this experiment adequately?
Laboratory Report Grading Rubric
Title of Report:
Grading Rubric Point Allocation for Lab Reports
Student learning in DL science courses with home based lab kits is at least equivalent to and usually a little better than in face-to-face courses with a campus based lab.
Valid assessment can be achieved via institution exams.
Feedback and Suggestions are Welcome!