critical thinking in medical education assessing what we mean and what we know l.
Skip this Video
Loading SlideShow in 5 Seconds..
Critical Thinking in Medical Education: Assessing What We Mean and What We Know PowerPoint Presentation
Download Presentation
Critical Thinking in Medical Education: Assessing What We Mean and What We Know

Loading in 2 Seconds...

play fullscreen
1 / 32

Critical Thinking in Medical Education: Assessing What We Mean and What We Know - PowerPoint PPT Presentation

  • Uploaded on

Critical Thinking in Medical Education: Assessing What We Mean and What We Know. Ed Krupat, PhD Director, Center for Evaluation Medical Education Grand Rounds Dec. 5, 2008. Components of Talks that I Have Liked. 1. A bit of context or framing

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Critical Thinking in Medical Education: Assessing What We Mean and What We Know' - Samuel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
critical thinking in medical education assessing what we mean and what we know

Critical Thinking in Medical Education:Assessing What We Mean and What We Know

Ed Krupat, PhD

Director, Center for Evaluation

Medical Education Grand Rounds

Dec. 5, 2008

components of talks that i have liked
Components of Talks that I Have Liked

1. A bit of context or framing

2. A touch of theoretical/conceptual background

3. A good portion of research and data

4. A bounty of concern for practical application

5. A strong dose of provocative discussion

context the origins of these efforts
Context: The Origins of These Efforts
  • Derek Bok--to--Richard Hersh
    • Evaluation of programs
  • Confusion-to-clarity-to-confusion-to-???
  • Why is critical thinking so important
    • In life
    • In medicine
  • Desire to be data-driven
conceptual analysis just what is critical thinking
Conceptual Analysis: Just What Is Critical Thinking??
  • Overlapping concepts
    • Analytic reasoning
    • Problem solving
    • Decision making
    • Clinical/diagnostic reasoning/judgement
    • Habits of mind
    • Meta-cognition
    • Adaptive expertise
some definitions critical thinking is
Some Definitions:Critical thinking is…
  • ...the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication as a guide to belief and action. (Scriven, 1996)
  • …the art of thinking about your thinking while you are thinking in order to make your thinking better: more clear, more accurate, and more defensible. (Paul et al, 1989)
research data
Research & Data
  • Two projects being conducted simultaneously

1.Qualitative: Do physicians agree about just what critical thinking is?

2.Quantitative: Can we assess critical thinking among medical students?

what is critical thinking the responses of physicians
What is Critical Thinking: The Responses of Physicians
  • Survey of practicing MD faculty at 5 medical schools
    • HMS, UCSF, Case Western, U Mass, Baylor (total n=73)
    • Convenience sample
two tasks
Two Tasks
  • Define critical thinking
  • Think of a clinical scenario in which critical thinking was important
    • describe it
    • state what a good critical thinker would do or say in that situation
    • state what a poor critical thinker would do or say
    • state how the outcome would differ if one or the other would have been the physician involved
coding of the definitions 1
Coding of the Definitions. 1
  • Definition specifically included:
    • Collection of information/data: 27%
    • Making sense of information: 96%
    • Utilization for decision making: 74%
    • Utilization for action: 14%
    • Specific ties to medical context: 43%
    • Necessity of building upon knowledge base: 27%
coding of the definitions 2
Coding of the Definitions. 2
  • Critical thinking characterized as
    • A process
    • A skill or ability
    • A disposition
critical thinking as a process
Critical Thinking as a Process
  • …a process of reflective reasoning that uses objective evidence, a deliberate weighing of options and alternatives, and clinical judgement and experience to guide decision making.
  • …the process by which one is able to rationally acknowledge different choices, processes and outcomes in the clinical encounter.
  • a process in which problems are being analyzed from different angles and connected to pre-existing knowledge before any conclusions are being drawn

44% of all responses

critical thinking as a skill or ability
Critical Thinking as a Skill or Ability
  • …the ability to think through a problem using reasoning. Also the ability to judge the credibility of sources.
  • …the ability to rigorously weigh the validity of evidence and then to effectively synthesize this evidence to reach a clinical decision.
  • …the ability to effectively problem solve using known data or under conditions of uncertainty.

50% of all responses

critical thinking as a disposition
Critical Thinking as a Disposition
  • …careful attention to what you know, vigilance for what you do not, and the courage to question both of the above categories.
  • …thinking about an topic, issue, or challenge in a way that sets aside my immediate ’gut’ response,so that I can be open and reflective to other possible ways of viewing the challenge…
  • …thinking deeply, keenly, flexibly, openly, reflectively, with an awareness of self and others, with attention to what is known and unknown, and with humility.

10% of all responses

clinical situations an interim report
Clinical Situations: An Interim Report
  • Looking for
    • What situations, choices, challenges differentiate critical and non-critical thinkers
    • What is it that critical thinkers do or say
    • What is it that non-critical thinkers do or say
    • What outcomes differ when critical thinking is or is not in evidence
  • Bottom line:
    • What are the key differences in the thoughts, words, and actions of critical thinkers
clinical contexts and tasks
Clinical Contexts and Tasks
  • Not surprising
    • Diagnosis and treatment
  • Surprising
    • Collaboration and interpersonal issues
      • How to protect patient’s rights and autonomy
      • How to deal with difficult patient requests
      • How to provide patient with appropriate options
the differences
The Differences
  • Critical thinkers:
    • Do more data gathering
    • Avoid premature conclusions
    • See inconsistencies in information
    • Utilize knowledge more extensively and explicitly to make decisions
    • Are aware of limitations and doubts
    • Monitor and evaluate their own decisions
    • Involve patients more fully
    • Provide patients with options
a few conclusions many questions
A Few Conclusions, Many Questions
  • Physicians are not necessarily all speaking the same language when they discuss critical thinking with one another
  • Is critical thinking a skill, a process, or a way of looking at the world?
    • Are skill and disposition both necessary?
    • Do you teach/encourage/foster skills in the same way as dispositions???
    • Do you assess skills in the same way that you assess dispositions?
  • Where does building upon a knowledge base come in?
how do we assess critical thinking
How do we assess critical thinking?
  • Collegiate Learning Assessment (CLA)
    • Used in over 200 colleges and universities
    • Assesses higher order outcomes
    • Students work on 90 minute performance tasks
    • System has been devised to generate reliable scores
  • Question: Is this a valid measure of critical thinking for medical students?
catfish one of the performance tasks used
Catfish: One of the Performance Tasks Used
  • Grotesquely mutated catfish has been found in the local lake that supplies the town’s water
  • You will serve on the mayor’s advisory panel
  • You are provided with 6 documents to read
    • Newspaper article
    • Editorial by environmental activist
    • Radio interview with a biologist from a nearby college
    • State report on water testing from lake
    • Area map
    • Journal article about similar discoveries
  • Open-ended written questions ask students to:
    • Identify main hypotheses to explain phenomenon
    • Identify strengths and weakness of each
    • State and defend most likely explanation
    • Suggest course of action
  • Evaluation of evidence
    • What is relevant, what is valuable
  • Analysis and synthesis of evidence
    • Connections, inconsistencies, flaws in reasoning
  • Conclusion drawing
    • Acknowledging alternatives and options
  • Presentation of arguments
    • Concise, evidence-based, logically structured
research design
Research Design
  • Recruit multiple schools
  • Recruit and compare students at two (or more) points in time
    • Entering Year 1 students took two tasks
    • Students at end of Year 3 took one task
  • Look for associations between CLA scores and other performance indicators
    • MCAT scores
    • Gender
    • Age
    • English as first language
    • Step scores
  • Compare scores between first and third year students
analysis problems
Analysis Problems
  • Among year 1 students
    • scores on the two tasks not as highly correlated as would be expected
    • Brain Boost task always given first
      • Year 1 students spend consistently less time and do consistently worse on Catfish (always second task) than on Brain Boost
      • Among Year 3 students--who only took one task, either Brain Boost or Catfish--students take more time and perform better than on Brain Boost
  • Small n’s
findings 1
Findings 1.
  • Medical school students have good critical thinking skills at baseline
    • Across the 3 schools, year 1’s
      • fell at the 87th percentile among graduating seniors nationally
      • were 1.25 standard deviations above the national mean
  • Is that good news or, with selectivity of the medical schools, should we expect to be even higher??
table 2 correlations among tests and demographic variables
Table 2. Correlations among tests and demographic variables

Significance codes: * p<.05, ** p<.01, *** p<.001

Note: Significance not adjusted for multiple significance tests

findings to come
Findings to Come
  • What is the correlation between CLA scores and
    • Step II CK
    • Step II CS
    • HMS Comprehensive Exam
what have we learned
What have we learned?
  • Critical thinking has many dimensions, and physicians may be talking past one another if they don’t share meaning when they speak.
  • Critical thinking can manifest itself in how to deal with patients, not just in generating diagnoses and treatment plans.
  • Critical thinking involves being
    • Sensitive
    • Inclined
    • Able

(Perkins & Ritchhart 2002)

what have we learned 2
What have we learned? 2
  • Medical students enter school with solid critical thinking skills
  • Critical thinking is somewhat, but not greatly, associated with other measured outcomes
  • It is still unclear the extent to which medical education fosters or hinders critical thinking
  • We have more questions at the end of this process than we did than when it began, but the questions are
    • more sophisticated
    • more tied to possible action
some big questions
Some Big Questions
  • If we think that critical thinking is so important, why don’t we explicitly design the curriculum to assure that we “teach” it?
  • If we wanted to do so, how and when would it be taught?
  • Should measures of critical thinking, as a skill or disposition, be built into our admissions screening process?
  • Should critical thinking be a core competency that is assessed throughout the curriculum? as a graduation requirement? As a requirement for licensure?