ruben j echemendia n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Neuropsychological evaluation of sports concussion/ tbi : Historical and psychometric considerations PowerPoint Presentation
Download Presentation
Neuropsychological evaluation of sports concussion/ tbi : Historical and psychometric considerations

Loading in 2 Seconds...

play fullscreen
1 / 25

Neuropsychological evaluation of sports concussion/ tbi : Historical and psychometric considerations - PowerPoint PPT Presentation


  • 148 Views
  • Uploaded on

Ruben J. Echemendia . Neuropsychological evaluation of sports concussion/ tbi : Historical and psychometric considerations. FDA Workshop Assuring the Safety and Effectiveness of Seizure, Cognitive Function, and TBI/Sports Concussion Diagnostic Devices Washington, DC: June, 2011. Disclosure.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Neuropsychological evaluation of sports concussion/ tbi : Historical and psychometric considerations' - winona


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
ruben j echemendia
Ruben J. Echemendia

Neuropsychological evaluation of sports concussion/tbi:Historical and psychometric considerations

FDA WorkshopAssuring the Safety and Effectiveness of Seizure, Cognitive Function, and TBI/Sports Concussion Diagnostic Devices

Washington, DC: June, 2011

disclosure
Disclosure

Dr. Echemendia is a paid consultant to:

  • The National Hockey League
  • Major League Soccer
  • US Soccer Federation
historical background
Historical Background
  • Neuropsychological assessment has been a key component in evaluating/managing TBI for decades.
  • Neuropsychological evaluations have become a key tool in the evaluation of MTBI since mid 70’s.
  • Focus on functional vs. structural assessment.
  • NP assessment of sports concussions began in mid 80’s.
background
Background
  • Sports Laboratory Assessment Model
  • Pre-post injury evaluations
  • Paper & Pencil Assessment Batteries
  • Computerized Batteries
psychometric considerations
Psychometric Considerations
  • Validity vs. Reliability
  • Validity
    • Does the test measure what it is supposed to measure?
    • Construct Validity –Operationalization of the construct, measurement of the construct
    • Cognitive tests DO NOT measure concussion!
    • Assess cognitive domains that are believed to be affected by concussion
  • Convergent/Divergent Validity
    • Does the test correlate with other known measures of the construct?
    • Does the test not correlate with domains that it is not supposed to measure Face Validity
  • Face Validity
    • Does the test appear to be measuring what it is supposed to measure
psychometric considerations1
Psychometric Considerations
  • Sensitivity vs. Specificity
  • Sensitivity: Is the measure “sensitive” in detecting the construct being measured?
    • Do concussed individuals score lower than non-concussed individuals?
    • What proportion of concussed individuals are identified by the test? Positive Predictive Power
    • What proportion of Non-concussed individuals are identified by the test? Negative Predictive Power
hypothetical 2x2 validity matrix

Abnormal NP Scores

Normal NP Scores

150

0

Concussed

150

Hypothetical 2X2 Validity Matrix

0

150

150

Normal

300

hypothetical 2x2 validity matrix1

Abnormal NP Scores

Normal NP Scores

120/ 80%

30/ 20%

Concussed

150

True Positive

False Negative

Hypothetical 2X2 Validity Matrix

60/ 40%

90/ 60%

150

Normal

True Negative

False Positive

300

psychometric considerations2
Psychometric Considerations
  • Sensitivity vs. Specificity
  • Specificity: Is the measure “specifically” measuring the identified construct and not others?
  • Concussion vs ADHD, Depression, Anxiety, Fatigue, etc.
  • Ideal: 100% Sensitivity – 100% Specificity
  • Real: NOT!
  • Sensitivity vs Specificity trade-off
  • Typically as sensitivity increases, specificity decreases
  • The user must decide on what side to err.
psychometric considerations3
Psychometric Considerations
  • Reliability – Estimating Error
  • Can we expect to get the same results when the test is given….
    • To the same person an infinite number of times? (Confidence Intervals)
    • To the same person under different circumstances? (Sideline vs. Office)
    • To the same person at different points in time?
    • To the same person by different test administrators?
    • In different/alternate forms?
paper pencil tests
Paper & Pencil Tests
  • Selected from tests already in use and validated for assessing MTBI
  • Test Requirements:
    • Short
    • Easily administered by non-NPs
    • Easily scored
    • Sensitive to changes in specified domain
    • Acceptable psychometrics
    • Alternate forms.
  • Assess several cognitive domains.
  • Typical Tests: HVLT. BVMT. Trails, Digit Span, SDMT, Verbal Fluency, LNS, Cancellation tests.
paper pencil tests cont
Paper & Pencil Tests Cont.
  • Advantages:
    • Direct observation of player behavior
    • Direct monitoring of effort/motivation & task performance
    • Superior assessment of memory
    • Broad normative db with multiple samples.
  • Disadvantages:
    • Labor intensive (Expensive)
    • Variability in standardized admin
    • Less reliable assessment of info proc speed.
    • Tests must be hand scored
computerized batteries
Computerized Batteries
  • Two approaches:
    • Adapt existing P&P measures into computer platform
    • Measure multiple cognitive functions (memory, proc speed, reaction time, etc.)
  • Develop tests de novo for computers
    • Limited assessment of domains (largely RT, Info Proc Speed)
  • New Instruments are being developed rapidly
  • Existing Instruments: ImPACT, ANAM Sports, HeadMinders CRI, CNS Vital Signs, CogSport.
computerized tests cont
Computerized Tests Cont.
  • Advantages:
    • Can be used in groups (Less costly)
    • Standardized admininistration
    • Immediate scoring/data capture
    • Finer assessment of RT/Proc Speed (altho other problems)
    • Multiple languages
    • Theoretically infinite alternate forms 
computerized tests cont1
Computerized Tests Cont.
  • Disadvantages:
    • Less complete assessment of memory
    • Loss of observational data
    • Less control of effort/motivation
    • Diminished ability to assess understanding/track task performance
    • Computer-specific measurement issues
    • Data loss
hybrid approaches
Hybrid Approaches
  • Combine strengths of both P&P and Computerized tests
  • Once adequate normative data have been established….
  • Computer for baseline testing
  • Computer plus P&P post-injury
sports mtbi specific issues
Sports MTBI specific issues

Phase of Evaluation

Baseline

Acute – On Field Management

Sub acute – Office/Training Room

Return to Play/Work/School

Long Term Follow-up

sports mtbi specific issues1
Sports MTBI specific issues
  • Baseline Testing – Is it necessary/helpful?
  • Theoretically designed to minimize error due to intra-individual comparison
  • Does it do this?
  • Does it increase sensitivity/specificity?
  • Does it introduce additional sources of error?
  • What about multiple post-injury tests?
  • How often should it be repeated?
  • Age dependent?
sports mtbi specific issues2
Sports MTBI specific issues
  • Test – Retest Reliability
  • Significant variability across studies
  • What test-retest intervals should be used?
  • Reliability estimates for multiple tests
sports mtbi specific issues3
Sports MTBI specific issues
  • Practice Effects
  • Content Practice vs. Procedural
  • Use of Alternate Forms
  • Reliability/Validity of Alternate Forms
  • Are practice effects bad?
  • How to control for / appropriately use practice effects?
    • RCI, regression based estimates
sports mtbi specific issues4
Sports MTBI specific issues
  • Motivation/Effort
  • Unique aspect of sports was the assumption that players are motivated to perform well.
  • Different motivation as different phases of assessment
  • Baseline vs. Post-injury
  • Individual vs. Group
  • Identification of poor effort
    • Test specific indices
    • Outlier identification
user qualifications
User Qualifications
  • Although not the responsibility of the FDA, clear delineation of user qualifications for administering and interpreting tests is critical.
  • Failure to do so may lead to scope of practice challenges, medico-legal quandaries and poor practice.