workgroup 5 pt provider assessment of laboratory performance n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Workgroup 5: PT Provider Assessment of Laboratory Performance PowerPoint Presentation
Download Presentation
Workgroup 5: PT Provider Assessment of Laboratory Performance

Loading in 2 Seconds...

play fullscreen
1 / 22

Workgroup 5: PT Provider Assessment of Laboratory Performance - PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on

Workgroup 5: PT Provider Assessment of Laboratory Performance. Mr. Daniel Tholen Dan Tholen Statistical Consulting Traverse City, Michigan, USA. Workshop Purpose. Present issues where there are differences in different regions where PT is common Guide for development in other regions

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Workgroup 5: PT Provider Assessment of Laboratory Performance' - braima


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
workgroup 5 pt provider assessment of laboratory performance

Workgroup 5: PT Provider Assessment of Laboratory Performance

Mr. Daniel Tholen

Dan Tholen Statistical Consulting

Traverse City, Michigan, USA

Wkgp 5 -- Subgp 1

workshop purpose
Workshop Purpose
  • Present issues where there are differences in different regions where PT is common
  • Guide for development in other regions
  • Deliberate redundancy and overlap of questions
  • Discuss the issues
  • Reach consensus or identify differences
  • Report to the main group and journal

WORKGROUP 5

slide3
5-1-1a: WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF PT/EQA PROGRAMS THAT ARE MANAGED FOR EDUCATIONAL AND/OR REGULATORY PURPOSES?
  • Threats of punishment cause a change in the way laboratories handle interlaboratory comparison samples.
  • What resources are needed for educational activities?

Wkgp 5 -- Subgp 1

5 1 1b are there some ways to satisfy both sets of needs
5-1-1b: ARE THERE SOME WAYS TO SATISFY BOTH SETS OF NEEDS?
  • Detection of poor performers
  • Elimination of bad performers
  • Elimination of bad performance
  • Information for lab self-improvement
  • Information for method improvement
  • Please produce a short list of the relative features, and recommendations on how to “have it both ways”.

Wkgp 5 -- Subgp 1

5 1 2a what are the considerations for determining when test performance can be graded
5-1-2a: WHAT ARE THE CONSIDERATIONS FOR DETERMINING WHEN TEST PERFORMANCE CAN BE GRADED?
  • In USA, CLIA requires “all tests” to be included in PT and graded appropriately.
  • “Regulated” vs. “Unregulated” analytes
  • “Graded” vs. “ungraded” analytes
  • Overall performance assessment

Wkgp 5 -- Subgp 1

slide6
5-1-2b: SHOULD EVERY TEST BE GRADED, OR ARE THER SOME TESTS FOR WHICH PT/EQA IS PREMATURE, REDUNDANT, OR UNNECESSARY?
  • What are the concerns for determining when an analyte is ready for grading, and what are the concerns for determining there is no need for PT?

Produce a list of concerns and analytes that could be excluded from PT.

Wkgp 5 -- Subgp 1

5 1 2c are there objective criteria for making this decision
5-1-2c: ARE THERE OBJECTIVE CRITERIA FOR MAKING THIS DECISION?
  • For example, statistics such as
    • Interlab agreement, all results and by group.
    • Intermethod agreement
    • Number of laboratories participating
    • Likely proportion unacceptable

Or: 2+ PT organizers grading the analyte?

Wkgp 5 -- Subgp 1

5 2 1a what performance measures are appropriate for quantitative and qualitative tests
5-2-1a: WHAT PERFORMANCE MEASURES ARE APPROPRIATE FOR QUANTITATIVE AND QUALITATIVE TESTS?
  • Accuracy (current)
  • Short term precision (repeatability)
  • Long term reproducibility
  • Uncertainty
  • Calibration or linearity
  • Knowledge / interpretation
  • Other?

Wkgp 5 -- Subgp 2

5 2 1b what statistical tools can be used to measure these characteristics
5-2-1b: WHAT STATISTICAL TOOLS CAN BE USED TO MEASURE THESE CHARACTERISTICS?
  • For measures other than accuracy, what would be appropriate statistical techniques?
    • Youden design
    • Repeat sample design
    • Linear design
    • ?

Please give a list, with preference order.

Wkgp 5 -- Subgp 2

5 2 2a what characteristics of pt eqa performance can be evaluated
5-2-2a: WHAT CHARACTERISTICS OF PT/EQA PERFORMANCE CAN BE EVALUATED?
  • This is supplemental to the previous question. Can PT be used to test skills such as interpretation or some aspects of handling?
  • If so, how?

Wkgp 5 -- Subgp 2

5 2 2b is it possible to evaluate a laboratory s interpretation of test results
5-2-2b: IS IT POSSIBLE TO EVALUATE A LABORATORY’S INTERPRETATION OF TEST RESULTS?
  • Provide a list of the best-supported ideas

Wkgp 5 -- Subgp 2

5 3 1a how should performance goals for laboratories be determined
5-3-1a: HOW SHOULD PERFORMANCE GOALS FOR LABORATORIES BE DETERMINED?
  • Relative to others (SD, percentiles, rank)
  • Expert Opinion – medical needs
  • State of the Art
  • Historical performance
  • Other

Wkgp 5 -- Subgp 3

5 3 1b should performance be measured relative to other laboratories or with objective goals
5-3-1b: SHOULD PERFORMANCE BE MEASURED RELATIVE TO OTHER LABORATORIES, OR WITH OBJECTIVE GOALS?
  • Relative to Others (common in EQA)
    • Z score
    • Percentiles
  • Objective Goals (discouraged in CLIA)
    • Fixed limits
    • Percentage limits

Wkgp 5 -- Subgp 3

5 3 2a what are the considerations for determining that a laboratory s performance is unacceptable
5-3-2a: WHAT ARE THE CONSIDERATIONS FOR DETERMINING THAT A LABORATORY’S PERFORMANCE IS UNACCEPTABLE?
  • Accreditation or regulatory requirements
  • Use PT alone, or with what other data?

Wkgp 5 -- Subgp 3

slide15
5-3-2b: SHOULD IT BE BASED ON A SINGLE TEST RESULT, A SET OF RESULTS IN A SINGLE TEST EVENT, OR RESULTS ACROSS SEVERAL TEST EVENTS?
  • CLIA criteria:
    • 80% of samples for each analyte
    • 80% of samples for each subspecialty
    • 2 out of 3 consecutive PT events

NATA: Accuracy and Repeatability w/Youden design. No carryover.

Wkgp 5 -- Subgp 3

5 3 2a should performance goals be the same for all types of laboratories
5-3-2a: SHOULD PERFORMANCE GOALS BE THE SAME FOR ALL TYPES OF LABORATORIES?
  • If PT goals are based on medical need, is it appropriate to have different criteria for different testing situations?
  • How would performance needs be defined?

Wkgp 5 -- Subgp 3

5 4 1 what are the advantages and disadvantages of blind pt eqa
5-4-1: WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF “BLIND” PT/EQA,?
  • Research findings
  • Oversight concerns
  • Designs where multiple pools are tested repeatedly
  • Please report any shared group opinions

Wkgp 5 -- Subgp 4

5 4 2 what are the criteria that should be used to determine the frequency of pt eqa
5-4-2: WHAT ARE THE CRITERIA THAT SHOULD BE USED TO DETERMINE THE FREQUENCY OF PT/EQA?
  • Practical concerns (cost, turnaround time)
  • Oversight concerns (undetected poor performance)
  • Frequency of calibration
  • Lab Workload

List criteria, preferences, objective measures

Wkgp 5 -- Subgp 4

slide19
5-5-1: WHAT FACTORS SHOULD BE CONSIDERED IN DEFINING PEER GROUPS IF SUCH GROUPS ARE USED IN DETERMINING ACCEPTABLE LABORATORY PERFORMANCE?
  • Tradeoff between group size and accommodating method bias
  • Material matrix effects vs. calibration and method standardization (traceability)
  • Need for verification?
  • Consider education and regulation

Wkgp 5 -- Subgp 5

slide20
5-5-2: WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF LABORATORIES RECEIVING IDENTICAL CHALLENGES IN EVERY TEST EVENT?
  • Do all labs need to see the same test items at the same time?
  • Advantages to alternate strategies?
  • Consider alternative design strategies (i.e. random samples from pools, multiple lots and labs pick, blind, split sample, etc.)

Wkgp 5 -- Subgp 5

summary
Summary
  • Purpose:
    • To discuss the issues
    • To reach consensus or identify differences
    • To report to the main group and journal
  • Discuss other questions or tie-in to morning workshops.

WORKGROUP 5

subgroup leaders and recorders
Dev Howerton

Tim O’Leary

Leigh Dini

Elizabeth Melnyk

Daniel Edson

Suzette Park

Mary Kimberly

Darshan Singh

Rex Astles

John Hancock

SUBGROUP LEADERS AND RECORDERS

WORKGROUP 5