Rubric based assessment of critical thinking skills l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

Rubric-based Assessment of Critical Thinking Skills PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Rubric-based Assessment of Critical Thinking Skills. Guy Hoelzer and Christie Howard Department of Biology UNR. or. More than “Just the Facts, Ma’am”: Assessment of Critical Thinking for Biology Majors and Programs. Guy Hoelzer and Christie Howard Department of Biology UNR.

Download Presentation

Rubric-based Assessment of Critical Thinking Skills

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Rubric based assessment of critical thinking skills l.jpg

Rubric-based Assessment of Critical Thinking Skills

Guy Hoelzer and Christie Howard

Department of Biology



More than just the facts ma am assessment of critical thinking for biology majors and programs l.jpg

More than “Just the Facts, Ma’am”:Assessment of Critical Thinking for Biology Majors and Programs

Guy Hoelzer and Christie Howard

Department of Biology


Our motivating factors l.jpg

Our Motivating Factors

  • Critical thinking (CT) has been identified by the Biology faculty as one of the key things we want our majors to be able to do well upon graduation

  • CT is notoriously difficult to teach and assess

    • Not part of our normal course content

  • Emphasis on CT in the capstone course for Biology majors leaves the impression that they are not very good at CT and don’t really understand what we mean by it

4 indeed they don t seem to trust the instructor s explanation of ct l.jpg

4. Indeed, they don’t seem to trust the instructor’s explanation of CT

  • It seems too subjective, so the target is the mind of the instructor rather than anything of value to the student

  • It seems that every instructor means something different by CT, so it doesn’t seem like a real or particular kind of skill

  • Instructors too often pay lip-service to CT without giving it weight in grading

5 generic ct skills vs discipline specific skills l.jpg

5. Generic CT Skills vs. Discipline-Specific Skills?

  • We would like to generate majors who “think like a biologist”

  • On the other hand, CT skills ought to be generally useful in any intellectual endeavor

6 student assessment of ct skills is hard enough can we do programmatic assessment of ct l.jpg

6. Student Assessment Of CT Skills Is Hard Enough - Can We Do Programmatic Assessment Of CT?

We would like to engineer a curriculum that builds on student CT skills in an integrated and coherent fashion

We propose to l.jpg

We Propose To …

emulate the rubric-based institutional model developed at Washington State University


The ct rubric for biology students l.jpg

The CT Rubric for Biology Students

  • Will include the key components of CT identified by the Biology faculty

  • Will serve as a template that can be modified to meet the needs of specific courses

  • Will be a primary tool for assessing student work at two or more points within the major’s curriculum

    • Track development of the CT skills of individual students

    • Focus on an Intro course (Howard) and the capstone course (Hoelzer)

Developing the rubric l.jpg

Developing the Rubric

  • H&H will design a 1st draft of the rubric based on the relevant assignments in their respective courses

    • Students are asked to criticize scientific papers in each course

  • H&H will do a pilot study by using the1stdraft of the rubric to assess 20-30 sample student papers from each course

    • Does the rubric effectively discriminate student performance over the full range encountered in both courses?

A faculty retreat l.jpg

A Faculty Retreat

  • Introduce the rubric to the Biology faculty (from both UNR and TMCC) and solicit feedback at a faculty retreat

  • TMCC will be included to facilitate articulation for transfer students

    • Ideally, the rubric will be useful and ultimately used in TMCC courses as well as UNR courses

  • Outside evaluators (experts) will be present to contribute to the discussion and report to H&H on the efficacy of our process

    • Evaluators, like John Mahaffy, will be consulted throughout the duration of the project

Consistency in rubric scoring l.jpg

Consistency in Rubric Scoring

  • H&H will design a second draft of the rubric based on feedback from the retreat

  • Clarify and calibrate the rubric with an Inter-rater reliability (IRR) study

    • Each member of the IRR team will score the same set of 20-30 pieces of student work

    • Meet to discuss differences among raters and possible causes of those differences

  • H&H will again revise the rubric based on the IRR results to maximize repeatability and tune the scoring scale to student variances

Another faculty retreat l.jpg

Another Faculty Retreat

  • Explain the revised rubric to the faculty

  • Workshop in which all faculty will use the rubric to score student work

    • Faculty training

    • Another round of faculty input into rubric development

    • Effectively another (informal) IRR study

    • Fostering faculty “buy-in” on use of the rubric

A novel student irr study l.jpg

A Novel Student IRR Study

  • It is just as important that the meaning of the rubric be clear to students (consistently interpreted among students) as to the faculty

    • Student interpretations should be the same as the intended meaning

    • Students should recognize a consistent thread regarding CT across courses and instructors

  • A team of students will use the rubric to score anonymous student work in an IRR study

Applying the rubric l.jpg

Applying The Rubric

  • Sample work from the Intro and Capstone courses will be collected and assessed

  • As the project ages, we will collect repeated samples from individual students where possible

  • We will aim to codify a comparable use of the rubric in one or more 300-level courses from the Biology major’s curriculum and encourage use of our CT rubric wherever appropriate throughout the Biology curriculum

Work by the assessment team l.jpg

Work by the Assessment Team

  • Team will meet after each semester to assess at least 30 papers from each course

  • Each meeting will start with a small IRR study to align the scoring practices of team members

  • Each piece of student work will be scored by two team members, and discrepant scores will be resolved by a third team member

  • We have proposed to pay all participants in this project for 3 years

    • We hope that use of the rubric by the faculty for both student and programmatic assessment will become valued and routine, and easily incorporated into ordinary departmental activities when the grant has ended

Some planned data analyses l.jpg

Some Planned Data Analyses

  • Track patterns of growth in student CT skills

  • Compare performance in CT skills with other variables

    • E.g., GPA, number of CT-type Biology courses taken, number of CLA courses taken

  • Compare 1st generation college student performance with others

  • Compare transfer with non-transfer students

  • Compare assessed CT skills with the perceptions of our alumni and their employers

Refine our curricular map l.jpg

Refine Our Curricular Map

  • Obtain a more detailed understanding of the extent to which CT skills are taught and/or assessed in particular courses

  • Are there gaps in the curriculum allowing some students to miss regular attention to CT skills?

  • Use the results of assessments and analyses to suggest curricular modifications addressing problems revealed by the process

Our guiding principles faculty input and training l.jpg

Our Guiding Principles:Faculty Input and Training

  • Workshops and retreats focused on defining and assessing CT

  • Discussion involvement and presentations by outside experts

  • Well paid faculty will participate in this project from start to finish

    • Improved quality control

    • Faculty buy-in regarding use of the rubric and the value of programmatic assessment

Our guiding principles including tmcc l.jpg

Our Guiding Principles:Including TMCC

  • Engaging TMCC Biology faculty in this sort of project should:

    • Improve our understanding of pedagogy at TMCC

    • Improve understanding by TMCC faculty of our curricular expectations

    • Facilitate the transition of transfer students into the UNR Biology major

Our guiding principles communicate our process and outcomes to others l.jpg

Our Guiding Principles:Communicate Our Process and Outcomes to Others

  • Align efforts with the UNR Core Curriculum plans to assess general (not discipline-specific) CT skills

  • Inform Biologists at all other UCCSN institutions of our project, our rubric, and our results

  • Once our rubric is adopted by the department for use throughout the curriculum, it will be published on the departmental website

  • We will publish our process and analytical results in a journal specializing in educational research

    • E.g., The Journal of Research in Science Teaching

  • Login