standardized letters of recommendation n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Standardized Letters of Recommendation PowerPoint Presentation
Download Presentation
Standardized Letters of Recommendation

Loading in 2 Seconds...

play fullscreen
1 / 36

Standardized Letters of Recommendation - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Standardized Letters of Recommendation. Anna H. Messner, MD Professor & Vice Chair Program Director Otolaryngology/Head & Neck Surgery Stanford University. Application season. Why have letters?. Review of 966 LOR to General surgery program 24% were helpful in evaluation process

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Standardized Letters of Recommendation' - josette


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
standardized letters of recommendation

Standardized Letters of Recommendation

Anna H. Messner, MD

Professor & Vice Chair

Program Director

Otolaryngology/Head & Neck Surgery

Stanford University

why have letters
Why have letters?
  • Review of 966 LOR to General surgery program
  • 24% were helpful in evaluation process

Fortune JB. The content and value of letters of recommendation in the resident candidate evaluative process. CurrSurg 2002

why have letters the theory
Why have letters? The theory.
  • Information on past performance which may be a good predictor of future success
  • Unique features
  • Insight into typical performance
    • (as opposed to maximal performance)
  • Opportunity to explain a less than perfect application
  • Opportunity to promote students he/she envisions as becoming future leaders in field.
slide5
Medicine “seems to have taken up permanent residence along the shores of Lake Wobegon. All of the applicants are above average—way above.”

All are outstanding

Schneider A. Why you can’t trust letters of recommendation. Chron High Educ 2000

writing letters the basics
Writing letters- The basics

1. Introduction

2. Relationship to applicant

Length of time he/she has known subject

Context of relationship

Right to view waived

3. Describe subject’s “record,”

Brief summary of subject’s academic performance

Summary of clinical performance

4. Subject’s traits and performance are evaluated and compared to peers (+ unique features).

5. Summary sentence

ohns words letter
OHNS Words/letter
  • 763 letters
  • Mean = 329 (82-1297)
  • Chair of dept(290)vs others (350) (p<.001)

Letters

Messner A, Shimahara E Laryngoscope 2009

what we currently do
What we currently do…
  • Look for Standout adjectives (average 2.6 in OHNS letters)
    • star, excellent, superb, gifted, extraordinary, outstanding, unique, exceptional, unparalleled, superstar, finest, exquisite, stellar, sterling, phenomenal, ideal, exemplary
  • Grindstone adjectives:
    • Hardworking, resolve, conscientious, dependable, meticulous, thorough, diligent, dedicated, careful, responsible, reliable, tireless, solid
  • “Gets along with others”, “Team player”
  • “Compassionate, Caring, Kind, Empathy”
  • “Intelligent”
features of letters
Features of letters
  • Leadership potential
  • High match rank
  • Sense of humor
  • Final sentence: Intentionally ambiguous
    • Recommend him without hesitation vs recommend highly vs highest recommendation vs absolute highest recommendation
    • Strong recommendation vs very strong recommendation vs strongest recommendation
  • Compare letters from same letter writer
watch for doubt raisers
Watch for doubt raisers
  • “Technical skills were average.”
  • “I have discussed alternative career options”
  • “Made an effort to be an effective team member.”
  • “Average fund of knowledge”
  • “His score of 210 accurately reflects his knowledge level.”
doubt raisers
Doubt raisers

Doubt Raisers

  • “It absolutely amazes me how he can work with others.”
  • “He’s always trying.”
  • “I assure you that no person would be better for the job.”
typical letter analysis beginning end
Typical Letter analysis: beginning & end

Dear Program Director:

I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology.

Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April 2006. This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology.

Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck.

She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read.

In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program.

As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me.

Sincerely,

typical letter ignore duplication
Typical Letter- ignore duplication

Dear Program Director:

I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology.

Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April 2006. This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology.

Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck.

She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. Sheis hard working and while somewhat shy Jane is clearly well informed and well read.

In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program.

As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me.

Sincerely,

typical letter analysis standouts grindstone team player
Typical Letter analysis: standouts, grindstone, team player

Dear Program Director:

I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology.

Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April 2006. This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology.

Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellentknowledge of the head and neck anatomy and diseases that afflict the head and neck.

She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasantperson to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read.

In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program.

As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me.

Sincerely,

typical letter analysis doubt raisers
Typical Letter analysis: doubt raisers

Dear Program Director:

I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology.

Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April 2006. This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology.

Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck.

She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellentsense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read.

In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program.

As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me.

Sincerely,

typical letter
Typical Letter

Dear Program Director:

I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology.

Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April 2006. This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology.

Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck.

She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read.

In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program.

As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me.

Sincerely,

NOTE: nothing unique, interesting, no comparison to peers, no mention leadership potential, no humor, no match potential

letter of minimal assurance
Letter of Minimal Assurance

“I believe that Joe is a strong candidate for a residency position and that he will ultimately be an excellent otolaryngologist and contributor to medicine. He is passionately committed to this goal and he is driven by that commitment. He is a unique person, and one whom you will enjoy meeting and interviewing.

I would of course welcome a direct phone call regarding this letter.

Thank you for considering Joe for an interview position.”

current status
Current Status
  • Average program: 250 applications x 4 letters = 1000 letters
  • Average letter = 329 words
  • Time = 1 min/letter (without analysis)
  • 1000 min = 17 hours.
  • Result: Letters often not used to make 1st cut
  • Increased reliance on USMLE scores
  • And… many not read at all.
standardized letter of recommendation slor
Standardized letter of recommendation (SLOR)
  • Council of Emergency Medicine Residency Directors created SLOR
  • Successfully used since 1996
comparison of slor nlor
Comparison of SLOR & NLOR
  • Part I: Compared 20 SLOR and 20 NLOR
    • 7- point Likert-type scale
    • 4 raters (2 experienced, 2 inexperienced)
    • Results
      • Interrater reliability of SLOR = 0.97, NLOR 0.78
      • Average time to interpret SLOR = 16 sec, NLOR = 90 sec

Girzadas DV Jr, Harwood RC, Dearie J, Garrett S. AcadEmerg Med. 1998 Nov;5(11):1101-4

comparison of slor nlor1
Comparison of SLOR & NLOR
  • Part I: Compared 20 SLOR and 20 NLOR
    • 7- point Likert-type scale
    • 4 raters (2 experienced, 2 inexperienced)
    • Results
      • Interrater reliability of SLOR = 0.97, NLOR 0.78
      • Average time to interpret SLOR = 16 sec, NLOR = 90 sec
  • Part II: 207 SLOR/NLOR pairs
    • Results
      • 54% given same numeric value, 39% differed by one point, 6% differed by 2 points. Overall correlation was 0.58.

Girzadas DV Jr, Harwood RC, Dearie J, Garrett S. AcadEmerg Med. 1998 Nov;5(11):1101-4

em slor
EM SLOR
  • 173 surveys of PD organization (70% response rate)
  • “Compared with the narrative LOR, please rate the SLOR in terms of:”
    • Ability to discriminate differences between candidates
      • Better 75%, Same 18%, Worse 7%
    • Ease of reading and incorporating into ranking scheme
      • Better 84%, Same 12% Worse 4%
    • Ease of completion
      • Better 83%, Same 9%, Worse 4%
    • Would you like to continue using the SLOR?
      • Yes 90%, No 7%

Keim SM, et al. Acad Emerg Med. 1999 Nov;6(11):1141-6

advantages of slor
Advantages of SLOR
  • More consistent information
  • Less duplication- of contents of application and letters
  • Less time to complete & interpret
  • Information less dependent on letter writer (more fair)
      • (A superb letter writer may portray a “good” candidate in stratospheric terms, whereas a mediocre letter writer may portray a “superb” candidate less positively)
  • Still have opportunity to tell about unique features of applicant
disadvantages of slor
Disadvantages of SLOR
  • Potential loss of nuances
  • Much harder to make the average student look stupendous
  • Potential attending MD guilt
  • Don’t get to show off fancy Department stationary
    • But OK to add Logo, or put form on stationary
next steps
Next Steps
  • Open for comments until February 1, 2012
    • Send to amessner@ohns.stanford.edu
    • OPDO council will review all comments and revise proposed SLOR
  • Assuming response is positive:
    • Will post SLOR on SUO-AADO website
    • Email distribution to PDs and Chairs
    • Need Program director assistance to get the word out to faculty and applicants
    • Investigate information re SLOR in Oto journal
    • Start using in spring/summer 2012
    • Survey SUO/AADO/PD re use, then revise.
conclusions
Conclusions
  • Traditional letters have the potential to help with identifying top residency candidates– but often do not.
  • A SLOR has the potential to increase the value of LOR while saving time.
  • Please send all comments to:

amessner@ohns.stanford.edu