sources of outcome data n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SOURCES OF OUTCOME DATA PowerPoint Presentation
Download Presentation
SOURCES OF OUTCOME DATA

Loading in 2 Seconds...

play fullscreen
1 / 70

SOURCES OF OUTCOME DATA - PowerPoint PPT Presentation


  • 146 Views
  • Uploaded on

SOURCES OF OUTCOME DATA. Internal measures End of course & clerkship surveys End of year surveys Faculty survey Universal Student Rating of Instruction (USRI) Certifying exam scores Canadian Graduate Questionnaire. External measures MCC

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SOURCES OF OUTCOME DATA' - jalia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sources of outcome data
SOURCES OF OUTCOME DATA
  • Internal measures
    • End of course & clerkship surveys
    • End of year surveys
    • Faculty survey
    • Universal Student Rating of Instruction (USRI)
    • Certifying exam scores
    • Canadian Graduate Questionnaire
  • External measures
    • MCC
    • Resident program directors’ evaluation of graduates
    • NBME-Comprehensive Basic Science Exam
    • CaRMs
    • Alumni
    • LCME report
    • Alberta Universities/Colleges Graduate Employment Survey

A Jones, Associate Dean – University of Calgary

overall rating of first year courses
Overall Rating of First Year Courses

Excellent

V.Good

Good

Fair

Poor

mean scores on certifying evaluations system courses yr 1 sb vs cp
Mean Scores on Certifying Evaluations (System Courses-Yr 1; SB vs CP)

A Jones, Associate Dean – University of Calgary

mean scores on certifying evaluations system courses yr 2 sb vs cp
Mean Scores on Certifying Evaluations (System Courses-Yr 2; SB vs. CP)

A Jones, Associate Dean – University of Calgary

alberta learning graduate employment survey 2004 graduates from 2002 medicine
ALBERTA LEARNING GRADUATE EMPLOYMENTSURVEY 2004 GRADUATES FROM 2002 MEDICINE

Usefulness of Your Education in Achieving:

  • Research Skills 80%
  • Working with Others 97%
  • A Desire to Learn More 93%
  • Learn Independently 97%
  • Awareness of Ethical

Issues 97%

A Jones, Associate Dean – University of Calgary

alberta learning graduate employment survey 2004 graduates from 2002 medicine1
ALBERTA LEARNING GRADUATE EMPLOYMENT SURVEY 2004GRADUATES FROM 2002MEDICINE
  • Satisfaction with the quality of

teaching in your program? 100%

  • Satisfaction with overall quality

of your educational experience 100%

  • University of Alberta 83%

A Jones, Associate Dean – University of Calgary

alberta learning graduate employment survey 2004 graduates from 2002 medicine2
ALBERTA LEARNING GRADUATE EMPLOYMENT SURVEY 2004GRADUATES FROM 2002MEDICINE
  • I would recommend the same

program of study to

someone else. 100%

  • Satisfaction with

Relevance of Courses 96%

A Jones, Associate Dean – University of Calgary

overall quality of education at u of c by faculty satisfied or very satisfied
Overall Quality of Education at U of C by Faculty: % Satisfied or Very Satisfied

Data Source: 2002 Alberta Universities/Colleges’ Graduate Employment Survey re: 2000 Grads

preparedness for residency

Strongly Agree

PREPAREDNESS FOR RESIDENCY

Agree

No Opinion

Disagree

Strongly Disagree

1: I am confident that I have acquired the clinical skills required to begin a residency program

2. I have the communication skills necessary to interact with patients and health professionals

3. I have basic skills in clinical decision making and the application of evidence based information to medical practice

4. I have the fundamental understanding of the issues in social sciences of medicine

5. I have the ethical and professional values that are expected of the profession

6. I have the fundamental understanding of the basic disease mechanisms, clinical presentations and principles of diagnosis and management for common conditions

Data Source: Canadian Graduate Questionnaire 2005

i am satisfied with the quality of my medical education
“I AM SATISFIED WITH THE QUALITY OF MY MEDICAL EDUCATION”

Data Source: Canadian Graduate Questionnaire 2003, 2004 & 2005

A Jones, Associate Dean – University of Calgary

performance on national exams
Performance on national exams

A Jones, Associate Dean – University of Calgary

slide13

720 – U of C

888- Canadian Grads/Canadian Trained

A Jones, Associate Dean – University of Calgary

slide14

328 – U of C

111- Canadian Grads/Canadian Trained

slide16
CaRMS: PERCENT OF MATCHED STUDENTS MATCHING TO FIRST CHOICE DISCIPLINE IN 1ST ITERATION CLASSES 2001- 2005

Data Source: CaRMS

A Jones, Associate Dean – University of Calgary

slide17
RESIDENT DIRECTORS’ ASSESSMENT OF GRADUATES (PGY1) “Overall Performance - ability to function as a resident with a full workload”

Data Source: Program Directors’ Survey

Class 2000 N = 50 (71%); Class 2001 N = 45 (68%) Class 2002 N = 40 (57%); Class 2003 N = 79(90%)

Class 2004 N = 76 (82%)

A Jones, Associate Dean – University of Calgary

resident program directors assessment of 2006 graduates
Resident Program Directors’ Assessmentof 2006 Graduates

Data Source: Program Directors Survey

A Jones, Associate Dean – University of Calgary

resident program directors assessment of 2005 graduates
Resident Program Directors’ Assessmentof 2005 Graduates

Data Source: Program Directors Survey

A Jones, Associate Dean – University of Calgary

resident program directors assessment of 2004 graduates
Resident Program Directors’ Assessmentof 2004 Graduates

Data Source: Program Directors Survey

A Jones, Associate Dean – University of Calgary

overall opinion of the undergraduate medical education program alumni survey classes 1992 2002
OVERALL OPINION OF THE UNDERGRADUATE MEDICAL EDUCATION PROGRAM ALUMNI SURVEY CLASSES 1992-2002
  • 97% Satisfaction with the UME program at University of Calgary
  • 90% Felt prepared or very prepared for Post Graduate Training
  • 98% Would advise their child or child of a relative or friend interested in Medicine to apply to the University of Calgary

A Jones, Associate Dean – University of Calgary

slide24

Why Curriculum renewal is Important

“A curriculum is like water. It has the tendency to seek the lowest level of energy it can reach, and without constant renewal, it will stagnate and become putrid. To avoid stagnation alone is justification for action.”

Acad Medicine Sept 1998

A Jones, Associate Dean – University of Calgary

barriers to medical school curriculum changes
Barriers to Medical School Curriculum Changes

Listed by North American Academic Deans:

  • Already crowded curriculum
  • Inadequate funding
  • Faculty resistance
  • Professional ‘turf’ issues
  • Scheduling conflicts

Graber et al. Acad Medicine 1997

A Jones, Associate Dean – University of Calgary

curriculum a planned educational experience
Curriculum – A Planned Educational Experience
  • Define the outcome measures.
  • Create an evaluation system to be sure these outcomes are realized.
  • Develop the pathways to get to these outcomes.

Allan R Jones, MD FRCPC, Associate Dean, Undergraduate Medical Education

goals for a revised curriculum
Goals for a Revised Curriculum

A revised curriculum has to be consistent with available information on clinical problem solving and reflect basic principles of adult learning.

A Jones, Associate Dean – University of Calgary

slide28

Medical students don’t remember or can’t use the knowledge they learned in the traditional basic science courses because the knowledge is structured into mental organizations that are not useful in the clinic

Barrows, 1985

A Jones, Associate Dean – University of Calgary

problem based learning benefits
Problem Based Learning – Benefits
  • Activate prior knowledge
  • Learn in context of clinical problem
  • Interest in learning stimulated
  • Self directed learning encouraged
  • Life long learning encouraged

Schmidt - Norman

A Jones, Associate Dean – University of Calgary

problem based learning concerns
Problem Based Learning Concerns
  • Problem solving skills are not augmented
  • Significant gaps in knowledge occur
  • Incorrect integration of basic sciences
  • Tendency to engage in backward reasoning

Albanese; Mitchell

Academic Medicine

A Jones, Associate Dean – University of Calgary

clinical reasoning
Clinical Reasoning
  • Clinical Reasoning and clinical knowledge are interdependent.
  • Effective problem solving requires a large store of relevant knowledge.
  • Clinical expertise is linked to depth and organization of clinical knowledge.

A Jones, Associate Dean – University of Calgary

problem solving skills in medicine
Problem Solving Skills In Medicine

Research has proven that experts in specific domains learn knowledge and problem solving skills for each problem simultaneously. That is, knowledge acquisition and clinical reasoning go hand-in-hand.

Schmidt et al 1992

efforts to help students improve clinical reasoning
Efforts to Help Students Improve Clinical Reasoning

Education must focus on the development of adequate knowledge structures. Teaching, coaching, supervising must strongly encourage and nurture actual knowledge organization of the students.

knowledge keeps no better than fish

Knowledge keeps no better than fish

Alfred North Whitehead 1929

clinical reasoning and small group cases
Clinical Reasoning and Small Group Cases

It is useful to select one model of clinical reasoning and base the tutorial discussion on it. The precise model is less important than its generic use as a framework to structure the flow of discussion. It later serves as a fall-back strategy in complicated clinical situations.

structure of medical knowledge in memory categories and prototypes
Structure of Medical Knowledge in Memory Categories and Prototypes

Both medical textbooks and classroom teaching abound in the limitless presentation of detailed lists of disorders. More often, both fail to provide a categorization scheme that is best suited for their retrieval in a clinical problem solving situation.

Bordage

Med Educ 1984

types of curricula
Types of Curricula
  • Disciplinary
  • Systems-based
  • Problem-oriented
  • Clinical Presentations based
clinical presentation curriculum
Clinical Presentation Curriculum

Faculty

Identify

Represented by

Identify

Core Competencies for Clinical Presentation

Curriculum Committee

Clinical Presentation

Develop

Plans and Monitors Curriculum

Enabling Basic Science Objectives

Terminal Objectives

Schematic Problem Solving Pathway

Course Content

For the Process of

Together Represent

Teaching Methods

Clinical Reasoning

Graduation Competencies

Guidelines for

Learning Content

Evaluation

steps in development and dissemination of clinical presentation objectives
Steps in Development and Dissemination of Clinical Presentation Objectives

1. Selection of clinical problem.

2. Classification system developed to help organize knowledge needed to solve the clinical problem.

3. Key Features; Discriminating features identified of prototypic prevalent disorders.

steps in development and dissemination of clinical presentation objectives1
Steps in Development and Dissemination of Clinical Presentation Objectives

4. Objectives and problem solving schemes developed.

5. Distribution to Faculty for balanced input from teachers generalists, specialists, and biomedical scientists.

6. Endorsements of objectives.

steps in development and dissemination of clinical presentation objectives2
Steps in Development and Dissemination of Clinical Presentation Objectives

7. Dissemination of objectives.

8. Encouragement of implementation of objectives in teaching, learning, clinical practice and problem solving

9. Monitor and evaluate the translation of objectives and problem solving schemes into practice.

clinical reasoning1
Clinical Reasoning

Student Identifies Clinical Presentation

Broad Classification of Problem

Schematic Problem Solving Pathway

Identify Causal Alternatives and Discriminating Key Factors

Differential Diagnosis

Diagnosis

Management Plan

the scheme
The Scheme
  • Causal Categories- pre, post and renal causes of acute renal failure
  • Diagnoses- specific diagnoses for each causal category
  • Basic sciences- Integral part- Timely presentation of content
slide44

“Ask any physician of 20 years standing how he has become proficient in his art and he will reply, by constant contact with disease; and he will add that the medicine he learned in schools was totally different from the medicine he learned at the bedside.”

Wm. Osler 1932

bleeding tendency bruising
Bleeding Tendency/Bruising

Hx PE DDx Invest. NatHx Mgmt

General Objectives

Thrombocytopenia

Disordered Platelet Function

Congenital Coagulation Disorders

Acquired Coagulation Disorders

Vascular Abnormalities

slide46

W Surgery

V Student

U Radiology

T Psychiatry

S Physiology

R Pharmacology

Q Pediatrics

O Pathology

N Oncology

M Office of Medical Education/Informatics/Culture, Health and Illness

L Office of Medical Bioethics

K Obstetrics & Gynecology

J Neuroscience

I Microbiology

H Medicine

G Immunology

F Genetics

E Family Medicine

D Community Health Sciences/Nutrition/Prevention

C Biochemistry

B Anesthesia

A Anatomy

Natural History, Prognosis & Complications of Condition

Prevention, Treatment & Complications of Treatment

Bleeding Tendency/ Bruising

Physical Examination

Differential Diagnosis

History

a

Investigation

GeneralObjectivesThrombocytopeniaDisordered Platelet FunctionCongenital Coagulation DisordersAcquired Coagulation DisordersVascular Abnormalities

b

c

d

e

f

g

0 1 2 3 4 5 6

schematic problem solving pathway
Schematic Problem Solving Pathway

Bleeding Tendency/ Bruising

Clinical Presentations

Platelets

Coagulation

Vascular

Broad Classification of Problem

Causal Alternatives and Discriminating Key Factors

Decreased Number

Abnormal Function

Congenital

Acquired

Congenital

Acquired

Differential Diagnosis

slide48

Pharmacology(ASA, Heparin)

Histology(Bone Marrow)

Genetics(Hemophilia)

Basic Science Objectives for Bruising and Bleeding

Anatomy(Spleen)

Physiology(Hemophilia)

Pathology(Vessels)

Immunology(ITP, Vasculitis)

basic science or biomedical knowledge in the undergraduate program
Basic Science or Biomedical Knowledge in the Undergraduate Program
  • The purpose of basic science teaching is to provide a scientific foundation for tasks of clinical practice such as diagnosis and therapeutics. The essential challenge of balancing depth of understanding with breadth of coverage remains.
  • (See p. 35, Fig. 4.1)
slide50

W Surgery

V Student

U Radiology

T Psychiatry

S Physiology

R Pharmacology

Q Pediatrics

O Pathology

N Oncology

M Office of Medical Education/Informatics/Culture, Health and Illness

L Office of Medical Bioethics

K Obstetrics & Gynecology

J Neuroscience

I Microbiology

H Medicine

G Immunology

F Genetics

E Family Medicine

D Community Health Sciences/Nutrition/Prevention

C Biochemistry

B Anesthesia

A Anatomy

Natural History, Prognosis & Complications of Condition

Prevention, Treatment & Complications of Treatment

Bleeding Tendency/ Bruising

Physical Examination

Differential Diagnosis

History

a

Investigation

GeneralObjectivesThrombocytopeniaDisordered Platelet FunctionCongenital Coagulation DisordersAcquired Coagulation DisordersVascular Abnormalities

b

c

d

e

f

g

0 1 2 3 4 5 6

fever
Fever

Fever

< 2 weeks > 2 weeks

Infectious

Bacterial Viral

acute visual loss
ACUTE VISUAL LOSS

PreRetinal

  • Corneal edema (glaucoma)
  • Vitreous hemorrhage (Diabetes)

Retinal

  • Acute Macular Lesion (hemorrhage)
  • Retinal Detachment (spontaneous)
  • Retinal Artery Occlusion (carotid emboli)

Post Retinal

  • Optic Neuritis (MS)
  • Ischemic Optic Neuropathy (Temp. Arteritis)
  • Occipital Infarction/Hemorrhage

Chrichton, Verstraton, Fletcher

strengths of year i ii curriculum
Strengths of Year I-II Curriculum
  • Approaches to problem solving – clinical reasoning
  • Early clinical exposure
  • Small group teaching
  • Clinical correlation; patient presentations
  • Basic science integration with problem solving
  • IST
  • Medicine 440; elective time
  • Communication; physical examination

Allan R Jones, MD FRCPC, Associate Dean, Undergraduate Medical Education

weakness of year i ii curriculum
Weakness of Year I-II Curriculum
  • Exams not always reflective of teaching ‘emphasis’
  • Lack of pharmacology
  • Faculty not promoting core documents; teaching to objectives not always clear
  • Small group teaching variable
  • Problem solving with schemes course dependent

Allan R Jones, MD FRCPC, Associate Dean, Undergraduate Medical Education

slide56

The Curriculum

Number of weeks for First Year = 45 weeks

A Jones, Associate Dean

slide57

The Curriculum

A Jones, Associate Dean

teaching methods
Teaching Methods
  • Lectures for rapid acquisition of key content
  • Small group case based learning- In depth self-directed (or guided) learning- Review, reinforcement, practice and feedback- Problem solving, motivation, pertinence
  • Clinical correlation- Bedside sessions
clinical reasoning2
Clinical Reasoning

Clinical reasoning does not develop in isolation: it is associated with increasingly refined and elaborated medical knowledge. Problem solving is domain-specific and not generic, so the challenge for medical educators is not only to make explicit the process of reasoning but also to identify the necessary content.

Schmidt et al 1990

Kassirer 1995

usual sequence of instruction
Usual Sequence of Instruction
  • Presentation and scheme shown- Case based or otherwise
  • Lectures or PBL sessions planned- In depth knowledge and acquisition- Basic and clinical sciences
  • Small group sessions for reinforcement- Shorter case scenarios for review- Clinical correlation
advantages of new curricular structure
Advantages of New Curricular Structure

Curriculum

  • Courses will be linked to graduation objectives and UME program philosophy of teaching, learning and evaluation.
  • Linkage of courses will integrate CP better and reduce redundancies.

Task Force Report

advantages of new curricular structure1
Advantages of New Curricular Structure

Curriculum

  • Clinical presentation list will be revisited and clerkships will adopt appropriate presentations.

Task Force Report

scheme use reported by first second year students classes of 2007 and 2006
Scheme Use Reported by First & Second Year Students(Classes of 2007 and 2006)

Strongly Agree

Neutral

Strongly Disagree

Data Source: Classes 07 & 06 yr end CP curriculum evaluation

map of department involvement in ume curriculum courses in years i and ii
MAP OF DEPARTMENT INVOLVEMENT IN UME CURRICULUM*COURSES IN YEARS I AND II

*Department involvement of 20 hrs structural teaching time

number of hours taught in the undergraduate medical program years i ii gft and clinical faculty
NUMBER OF HOURS TAUGHT IN THE UNDERGRADUATE MEDICAL PROGRAM YEARS I-II GFT and CLINICAL FACULTY

*Dr Doran responsible for 200 hrs

learning methods used during years i and ii university of calgary undergraduate program
LEARNING METHODS USED DURING YEARS I AND II UNIVERSITY OF CALGARY UNDERGRADUATE PROGRAM

Small Groups/Bedside: 1/3

Large Groups/Classroom: 1/3

Self Directed Study: 1/3

A Jones, Associate Dean

slide67
SUMMARY OF STRUCTURED TEACHING HOURSUNDERGRADUATE MEDICAL EDUCATION PROGRAM YEARS I AND II 2004-2005

*Medicine 440, Summer Elective not included

A Jones, Associate Dean

ownership of the curriculum
Ownership of the Curriculum

"Faculty members own what is taught in the curriculum - they own the content. The Associate Dean and Curriculum Committee are responsible for the methods and the effectiveness. Faculty give the course, the Associate Dean has to give the degree.”

Academic Medicine Vol. 73, Pg. 54

the seven most dangerous words in medical education
The Seven Most Dangerous Words in Medical Education

“…but we’ve always done it this way”

Allan R Jones, MD FRCPC, Associate Dean, Undergraduate Medical Education

slide70

Prototype Timeline for a Major Educational Change

  • Decide what to do.
  • Build support for the idea.
  • Acquire resources.
  • Do it wrong the first time.
  • Do it wrong the second time.
  • Do it passably the third time.
  • Do it reasonably well the fourth time.

Friedman Acad Medicine 1993