Approaches to Assessing Critical Thinking

1. EDU Lunchtime Seminar Approaches to Assessing Critical Thinking 12 May 2000 Kam-Por Kwan x6287 etkpkwan@ Patrick Lai x6294 etktlai@

2. Seminar series • 12 May 2000: Approaches to Assessing Critical Thinking • 18 May 2000: Overcoming the Challenges in Developing Students’ Critical Thinking • 9 June 2000: Measuring Students’ Critical Thinking: Problems and Possible Solutions

3. Outline of today’s session • Overview of approaches to assessing critical thinking (CT): • Examples • Relative advantages and disadvantages • Appropriateness for different purposes of CT assessment • Applicability to your context • PolyU Special Interest Group for Teaching and Assessing CT

4. Purposes of CT assessments • Programme/subject evaluation: evidence of success in promoting CT • Authentic assessment: • evidence of effectiveness of CT instruction • motivating students to be better at CT • Diagnostic feedback on students’ CT level: • for designing instruction • for faciliating students’ growth & development • Research into CT instructions • Selecting students for admission

5. Definition of CT • Ennis (1993: 180): “Critical thinking is reasonable reflective thinking focused on deciding what to believe or do” • APA Delpi Report (1990): “Critical thinking is the process of purposeful, self-regulatory judgement. This process gives reasoned consideration to evidence, contexts, conceptualisations, methods and criteria.”

6. Deciding what to believe or do • Judge the credibility of sources • Identify conclusions, reasons, and assumptions • Judge the quality of an argument: acceptability of its reasons, assumptions, and evidence • Develop and defend an issue • Ask appropriate clarification questions • Plan experiments amd judge experimental designs • Define terms in a way appropriate for the context • Be open-minded • Try to be well-informed • Draw conclusions when warranted, but with caution

7. CT Dispositions and skills • Two components of critical thinking (Facione , 1996): • Affective dispositions: the necessary attitudes and habits in approaching problems, questions, and issues for CT to take place • Cognitive skills: the mental ability needed for critical thinking

8. Critical thinking dispositions Open-mindedness Inquisitiveness Systematicity Dispositions towards CT Cognitive maturity Analyticity Truth-seeking CT Self confidence

9. Critical thinking skills Inference Analysis Explanation CT Skills Interpretation Evaluation Self-regulation Source: Facione (1996)

10. Approaches to CT assessments • Students’ self-reported skills / gains in CT • Students’ performance on standardised published CT tests • Instructor-constructed CT tests • Student interviews / protocol analysis • Students’ performance on tasks requiring CT skills • performance assessment • rubric for assessing CT in students’ work

11. Self-reported skills /gains in CT • Students’ self reports on: • skills in CT • gains or progress in CT skills • Examples: • Items from CSEQ • Items from IDEA • Relatively easy and inexpensive to use • Less reliable and valid

12. Published CT tests • Standardised tests with well-established psychometric properties • Examples: • The California Critical Thinking Test: College level (CCTST) • Cornell Critical Thinking Test, Level Z (1985) • Watson-Glaser Critical Thinking Appraisal (1980) • Reliability and validity established , often with norms for comparison • Expensive to use, non subject-specific, implicit cultural and contextual assumptions

13. Instructor-constructed CT tests • Developed by instructor • Can be tailored for specific subject, context, or purpose • Relatively costly to design (both in time and effort), reliability and validity need to be determined, difficult to compare across groups or contexts

14. Interview or protocol analysis • Students given a problem to solve in an interview • to find out ways person understands a concept or solves a problem • Researcher analyses students’ explanation, and infers from that to form the researcher’s description • Result consists of categories which “describe different ways of understanding a concept or solving a problem” • These categories stand in a logical hierarchical relationship with one another and have implications for teaching

15. Interview method • Students given a problem to solve in an interview • They are asked to think aloud and tell the interviewer their thoughts as they are going through • Probing/Prompted questions • You mentioned about X. How does this occur? • Do these symbols have any meaning to you? • Why did you give up your original hypothesis?

16. Protocol / data analysis • Verbatim transcription of audio-taped interviews • Transcription printed and read through with a particular research question in mind • Descriptive words or phrases attached to short sections of the transcript as a potential category • Sections of the transcript that could fit into this potential category labelled by sub-categories • Content of one excerpt compared and contrasted with the content of another similarly labelled excerpt • Labelled excerpt organised into flow charts.

17. Performance assessments • Students’ written and / oral descriptions analysed by the Holistic Critical Thinking Scoring Rubric (HCTSR) (Facione and Facione, 1994) • HCTSR measures • critical thinking skills to reach a judicious judgment which include analysis, interpretation, evaluation, inference, explanation • assesses dispositions to pursue reasons and evidence fair mindedly and open-mindedly to reach sound, objective decision • HCTSR consists of four levels - level 4 signifies a critical thinker

18.           Assessing CT under the 3-P model

19.                  CT Assessment for different purposes

20. SIG for teaching & assessing CT • Sharing of resources and experiences in teaching and assessing CT through • e-newsletter • regular meetings • Collaboration in designing and evaluating CT instructions • trying out innovative methods for CT instruction • small-scale classroom research • Contact persons • Patrick Lai: etktlai@polyu.edu.hk x6294 • K. P. Kwan: etkpkwan@polyu.edu.hk x6287