Understanding and doing evidence based research in pathology
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Understanding and doing evidence based research in Pathology PowerPoint PPT Presentation


  • 49 Views
  • Uploaded on
  • Presentation posted in: General

Understanding and doing evidence based research in Pathology. I. The Question Randomized Controlled Trials (with thanks to Brian Haynes) Observational Trials (with thanks to Mitchell Levine). Goals of presentation. Describe study types Descriptive Analytic

Download Presentation

Understanding and doing evidence based research in Pathology

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Understanding and doing evidence based research in pathology

Understanding and doing evidence based research in Pathology

I.

The Question

Randomized Controlled Trials

(with thanks to Brian Haynes)

Observational Trials

(with thanks to Mitchell Levine)


Goals of presentation

Goals of presentation

  • Describe study types

    • Descriptive

    • Analytic

  • Discuss how to formulate a research question

  • Discuss the study designs that will answer the research question


Making evidence based decisions

Making evidence based decisions

  • Making decisions

    • What is this disease?

      • What additional tests should be done?

    • How will this disease behave?

    • What information do I need to provide for management purposes?

  • To make evidence based decisions, the research informing your decisions must provide the closest possible approximation to the truth


Research types clinical research

Research types (“clinical” research”)

  • Case report

  • Descriptive studies (Observational)

    • Case series

  • Analytic studies

    • measure associations

    • comparison group present

      • case control, cohort (Observational)

      • RCT (clinical trial)


The question

The question

  • PICO (T)

    • Population

    • Intervention

    • Comparison intervention

    • Outcome

    • (Time)

  • Structures the question


Rct question

RCT question

  • In patients with clinically and histologically node negative breast cancer, does sentinel node biopsy alone achieve the same survival as axillary lymph node dissection?

    • P = patients with clinically and histologically node negative breast cancer

    • I = sentinel node biopsy

    • C = axillary lymph node dissection

    • O = survival

NSABP B-32. Krag et al. Lancet Oncology 2010;11:927-33


Rct question1

RCT question

  • In patients with metastatic squamous cell carcinoma to cervical lymph nodes diagnosed on FNA, what proportion have positive test results when the FNA is tested for oncogenic HPV-DNA, E6 oncoprotein overexpression or p16/Ki-67 dual staining cytology with HPV triage? (A. Abdel-Mesih)

    • P = patients with metastatic squamous cell carcinoma to cervical lymph nodes diagnosed on FNA

    • I1 = E6 oncoprotein overexpression

    • I2 = p16/Ki-67 dual staining cytology with HPV triage

    • C = oncogenic HPV-DNA test

    • O = positive test


Observational question

Observational question

  • PICO

  • Replace “intervention” by “exposure”


Observational study single cohort case series

Observational study (single cohort/ “case series”)

  • Prognosis

  • What is the 5 year survival in women with breast cancer who have axillary lymph node metastases no larger than micrometastases or ITC’s?

    • P = women with breast cancer who have axillary metastases no larger than micrometastases or ITC’s

    • I/E = time

    • O = survival

    • T= 5 years

      Or

    • P = women with breast cancer

    • I/E = axillary metastases no larger than micrometastases or ITC’s

    • O = survival

    • T = 5 years


Understanding and doing evidence based research in pathology

  • What is the 5 year survival in women with breast cancer who have axillary lymph node metastases no larger than micrometastases or ITC’s?

  • Here there is no comparison group. But maybe what the patient really wants to know (patient centred or patient relevant outcome) is whether her survival is adversely affected by the presence of micromets or ITC’s compared to what her survival would be if she had negative nodes. You can infer this by seeing what the literature says in regards to node negative survival, or you could do a direct comparison


Observational study cohort with comparator group

Observational study (cohort) with comparator group

  • In women with breast cancer, do micrometastases or ITC’s in axillary lymph nodes decrease 5 year survival as compared to women with axillary node negative breast cancer?

    • P = women with breast cancer

    • I / E = axillary node micromets/ITC

    • C = negative axillary nodes

    • O = survival

    • T= 5 years

  • Here you have 2 groups

    • Group 1 survival

    • Group 2 survival


Observational study cohort with comparator group1

Observational study (cohort) with comparator group

  • Subgroup of DLBL, now called BCLU

    • Is BCLU a distinct entity other than on histology?

    • Plan was to look at clinical presentation and OS and DF survival. OS was the more important outcome, so it was made the primary outcome

  • Do patients with BCLU have decreased OS compared to patients with DLBL?

  • Among patients traditionally called DLBL , is BCLU histology associated with decreased overall survival compared to those with conventional DLBL histology? (L. Edwards)

    • P = patients traditionally diagnosed as DLBL

    • I / E = BCLU histology

    • C = DLBL histology

    • O = overall survival

  • Here again you have 2 groups


  • Observational study cohort with comparator group2

    Observational study (cohort)with comparator group

    • In patients with endometrioid carcinoma of the endometrium, does LVSI confer decreased survival?

    • LVSI is a “prognostic” factor

      • you are still dealing with 2 groups, one with and one without the prognostic factor: Among patients with endometrioid carcinoma of the endometrium, is survival decreased in those with LVSI, as compared to those without LVSI?

        • P =patients with endometrioid carcinoma of the endometrium

        • I / E = LVSI

        • C = no LVSI

        • O = survival (OS, Disease Specific Survival?)

        • T = ?


    Diagnostic testing 1

    Diagnostic testing (1)

    • In women with a papillary lesion on needle core biopsy, does using a diagnostic algorithm identify lesions that require surgical excision?

    • In women with a papillary lesion on NCB, what is the sensitivity and specificity of a histologic diagnostic algorithm for identifying ADH or worse lesion, as assessed from the surgical excision? (LJ Elavathil)

      • P = women with a papillary lesion on NCB

      • I = use of a diagnostic algorithm when reading the core (“histologic diagnostic algorithm”)

      • O = ADH or worse in the excision specimen


    Diagnostic testing 2

    Diagnostic testing (2)

    • In women referred to colposcopy with ASCUS or LSIL on screening Pap smear, what is the accuracy of p16/Ki-67 dual stain cytology for detecting histologically confirmed CIN 2+? (M. Schell)

      • P = women referred to colposcopy with ASCUS or LSIL on screening Pap smear

      • I = p16/Ki-67 dual stain cytology

      • O = histologically confirmed CIN 2+


    Studies of association

    Studies of association

    • The preceding questions asked whether the intervention or exposure was associated with the outcome

    • There is hierarchy of study design

      • RCT gives the most credible association of intervention with outcome

      • RCT cannot be done for

        • exposures that cannot be assigned, e.g. LVSI in tumors

        • studies of harm e.g. give DES to pregnant women to confirm suspected adverse effect on fetus

      • RCT may not be feasible if outcome is rare or requires prolonged time to occur

      • Designs that are more prone to bias may need to be used


    Randomized controlled trials

    Randomized controlled trials

    • You create the intervention

    • Randomization (if concealed) ensures groups differ only in the intervention

      • Both unknown and unmeasured important prognostic factors are likely to be similar among groups because of random allocation

      • Only way to control for unknown prognostic factors that can influence the outcome of interest


    Controlling for factors that influence outcome rather than do rct

    Controlling for factors that influence outcome rather than do RCT

    • Controlling for known factors (create subgroups or do statistical adjustment)

      • In women breast cancer, do micrometastases or ITC’s in axillary lymph nodes decrease 5 year survival as compared to women with axillary node negative breast cancer?

        • Tumor size, LVSI, hormone receptor status, treatment

      • In patients with endometrioid carcinoma of the endometrium, does LVSI confer decreased survival?

        • Stage

      • FNA HPV test positivity

        • Sequence of testing and sample adequacy

    • What about unknown factors?


    Evidence suggesting results from rcts are more valid than from observational studies

    Evidence suggesting results from RCTs are more valid than from observational studies

    • Example: vitamin E

      • cohort study (n >5000) showed significantly fewer deaths due to CAD

      • RCT (n>5000) showed no effect , and harm from high doses

    • Cochrane review of observational studies showed frequently worse prognosis (i.e. sicker patients) at entry among control patients

      • The result is biased from the beginning; intervention will appear more successful even if it is not


    Randomized controlled trials1

    Randomized controlled trials

    • Are they relevant to pathology?

      • Will you ever be the PI or design a RCT?

        • Yes, why not if the research question is relevant to you?

          • HPV testing for cervical cancer screening: you read the Pap smears, you have the right to evaluate the alternatives

      • Will you be a co-investigator?

      • Will you need to interpret the published study to decide to change or maintain current service provision?


    Why start with rct

    Why start with RCT?

    • Concepts in RCT’s applicable to all study designs

    • Concepts are easiest to understand in the RCT design


    Rct itemized checklist

    RCT: Itemized checklist

    • CONSORT statement

    • Parallel group randomized trials

      • Extension for trials assessing nonpharmacologic treatments


    Design issues

    Design issues

    • Defining the population

    • Defining the intervention

    • Generating randomization sequence

    • Defining the outcome (s)

    • Blinding

    • Sample size

    • ALL except randomization are pertinent to other study designs as well


    Understanding and doing evidence based research in pathology

    Sentinel node vs axillary dissection in breast cancer NSABP B-32Krag et al. Lancet Oncology 2010;11:927-33

    • Population

      • Inclusion criteria

        • Women with invasive breast cancer

        • Clinically negative nodes

        • Sentinel nodes negative on HE (and negative IHC when performed for confirmation of suspicious findings on HE)

      • Exclusion criteria


    Understanding and doing evidence based research in pathology

    NSABP B-32

    Women with breast cancer and clinically negative axillary nodes

    • Stratification

    • - Age ≤49, ≥50

    • cT ≤2, 2.1-4.0, ≥4.1 cm

    • Surgery (lumpectomy, mastectomy)

    R

    SLN only

    SLN + ALND

    SLN negative

    SLN negative

    • Follow to outcome


    Intervention nsabp b 32

    InterventionNSABP B-32

    • Intervention: sentinel node dissection, lymph nodes sectioned at 2mm, no routine IHC

    • Control: SLN dissection with ALND; and SN

      sectioned at 2mm, no routine IHC


    Randomization and intervention assignment

    Randomization and Intervention Assignment

    • Want every participant to have equal chance of being assigned to intervention or control, regardless of clinical status

    • Random assignment to group

      • Generate unpredictable allocation sequence

        • So that no one can predict what is the allocation assignment


    Understanding and doing evidence based research in pathology

    Simple Randomisation

    Unequal sample sizes if study small (<200)

    Unequal patients with important prognosticators: stratify


    Randomization

    Randomization

    • Generate unpredictable allocation sequence

    • Conceal this sequence until assignment

      • Call in to study centre √

      • Sealed envelopes


    Intervention and comparison

    Intervention and comparison

    • Should be indistinguishable in appearance, taste, etc

      • Easier to do in drug trial

      • May not be possible in surgical trial


    Outcome

    Outcome

    • NSABP B-32

      • primary outcome: overall survival

      • Secondary outcomes: DFS, second cancers, etc

    • Important to specify

      • Sample size must be sufficient to detect at least the primary outcome

      • Secondary outcomes more susceptible to false positive results (issue of p value and multiple comparisons)


    Defining the outcome

    Defining the Outcome

    • What is the outcome?

    • Patient important outcomes

      • Survival: OS? Disease specific?

    • How will you measure or identify the outcome?

      • Survival: clinic charts? Contact family doc/family? Death registry?

    • Adjudication for subjective outcomes

      • e.g. CIN


    Outcome1

    Outcome

    • Composite outcomes

      • e.g. CV death, stroke, MI, angina

      • Useful for increasing events, therefore decreasing sample size

    • UKPDS: 3867 DM2 x 10y (L.A. Moye. Multiple analyses in clinical trials. Springer, NY, 2003)

      • diabetes-related endpoints: sudden death, death from hypo/hyperglycemia, fatal /non-fatal MI, angina, heart failure, stroke, renal failure, amputation, vitreous hemorrhage, retinal photocoagulation, blindness in 1 eye, cataract

      • Intensive vs control Tx : 35.3% vs 38.5%, p=0.029

      • Difference 3.2% (retinal photocoagulation 2.7%)


    Surrogate vs major outcomes

    Surrogate vs major outcomes

    • Outcomes that are not themselves important to patients, but are associated with outcome that ARE important to patients

      • Bone density for fracture (not so good)

      • CIN 2+ for cervical invasive squamous carcinoma (OK)


    Blinding

    Blinding

    • Can potentially blind

      • Participants, health care providers, data collectors, outcome adjudicators, data analysts

    • Should blind if knowing allocation can cause differential reporting and search for outcomes, especially if outcomes are subjective

      • May not be needed for objective outcomes such as death

    • Prevent co-intervention, contamination


    Sample size

    Sample size

    • Small studies

      • Significant differences: the 2 groups may have different prognoses and therefore differences may be due to chance

      • If events are few and no significant difference, cannot exclude possibility of important difference

    • Need adequate sample size

      • Too small, problems as above

      • Too large, waste resources, potentially put patients at risk

      • Can calculate estimated sample size

        • Need to know baseline risk in control group, the clinically important difference one want s to see, required power, alpha


    Sample size nsabp b 32

    SAMPLE SIZE NSABP B-32

    • Trial was powered to test a difference in survival of 2% between the 2 groups at 5 years. 300 deaths were needed to trigger the overall survival analysis


    Statistical analysis

    Statistical analysis

    • How are you going to analyze your data once you get it?

      • Difference in proportions (what stats test?)

      • Difference in survival

        • Univariate (Kaplan Meier)

        • Multivariate adjusting for potential confounders (Cox proportional hazards)


    Some issues in reporting results

    Some issues in reporting results

    • What patients did you actually get?

      • Describe baseline characteristics (give p value?)

    • Effectiveness vs efficacy

      • NSABP B-32: “Analyses were done according to the random assignment of patients…” Effectiveness

      • drug trials

        • Effectiveness: Analysis performed according to assignment group

        • Efficacy: Analysis performed according to the intervention the patient actually received

    • Subgroup analyses

      • Numbers may be too small to detect real difference

      • Spurious significant p values

      • Rationale, limit number of analyses, a prior questions


    Additional issues

    Additional issues

    • Non inferiority vs superiority

      • NSABP B-32: non inferiority trial

        • “trial was designed to establish whether SLN resection achieves the same therapeutic outcomes as ALND”

        • “to test a difference in survival of 2% between the 2 groups”

        • Can never test for difference of 0 because sample size would be infinity; set mininum difference

        • Requires larger sample size than superiority trial

    • Interim analyses

      • Pre-set p values for each interim analysis; overall p value for the trial must be 0.05

    • Early stopping

      • To avoid toxicity or delayed benefit if one arm appears significantly superior to the other

      • Early results may reverse by the end of the trial


    Clinical vs statistical significance

    Clinical vs statistical significance

    • Weaver et al, NEJM 2011;364:412-21

      • Sentinel nodes examined at deeper levels with HE and IHC

    • Significant difference in overall survival (p=0.03) between patients with occult metastases and those without metastases

    • Adjusted HR 1.40 (95%CI 1.05-1.86)

    • 5 year OS

      • occult metastases 94.6%

      • no metastases 95.8%

    • “…differences…statistically significant, but relatively small…”

    • “Our findings argue against analysis of additional tissue levels or routine immunohistochemistry analysis for sentinel lymph node evaluation.”


    Generalizability external validity

    Generalizability / external validity

    • Are patients similar to the study participants?

    • Is the exposure similar?


    Observational studies case control

    Observational studies case control

    • Useful when event rare, prolonged delay between exposure and outcome

    • Identify by outcome

      • Patients with lung cancer vs patients without lung cancer

      • Control group: people in community?, hospitalized patients for orthopedic surgery?, patients with CAD?

  • Look for smoking exposure in both groups

  • Control for important confounders / matching

  • Analyze: is there is increased proportion of smokers among patients with lung cancer?


  • Understanding and doing evidence based research in pathology

    Hospitalized patients with lung cancer

    Patients hospitalized for orthopedic surgery

    Exposure: current

    or past smoker?

    Exposure: current

    or past smoker?

    *Adjust for important variable such as age, etc.


    Observational studies cohort

    Observational studies - Cohort

    • Start with exposure

      • DLBL and BCLU patients

    • Follow to outcome

      • Death due to any cause

    • Data can be gathered prospectively

      • Can control what data will be gathered

        • e.g. do c-myc testing in all patients as biopsies are received

    • Data can be gathered retrospectively

      • Data may be incomplete

        • E.g. c-myc testing done on some patients and report available; myc testing on others can still be done, but may not have adequate tissue in some


    Cohort

    Cohort

    Population

    Exposure (histology)

    DLBC

    BCLU

    Outcome :

    Overall survival

    Outcome :

    Overall survival


    Cohort itemized checklists

    Cohort: Itemized checklists

    • Strobe statement

      • Similar to Consort, but no randomization

      • “variables: clearly define all … exposures, predictors, potential confounders and effect modifiers”

    • REMARK (reporting recommendations for tumor marker prognostic studies)

      • Similar to Strobe, emphasis on describing the prognostic marker, defining other predictors of outcome


    Dealing with confounders

    Dealing with confounders

    • If a potential confounder / predictor is known and can be measured, the analysis requires either a statistical adjustment for the confounder / predictor or subgroup stratification


    Understanding and doing evidence based research in pathology

    Clinical characteristics


    Understanding and doing evidence based research in pathology

    Overall Survival for DLBL vs BCLU

    HR2.5 (95% CI 1.2-5.2)

    Adjusted for IPI and treatment


    Exposure and outcome why are case reports series less trustworthy

    Exposure and outcome: why are case reports / series less trustworthy?

    Outcome

    +

    -

    +

    Exposure

    -


    Exposure and outcome why are case reports less trustworthy

    Exposure and outcome: why are case reports less trustworthy?

    Ovarian serous carcinoma

    +

    -

    +

    STIC

    -


    Is association causation

    Is association causation?

    • Strength of association

    • Consistency

    • Specificity

    • Temporal relationship

    • Dose response

    • Experimental evidence

    • Analogy

    • Coherence

    • Biologic plausibility


    Strength of association

    Strength of association

    • Magnitude

    • Statistical significance

    • Precision


    Relative risk rate ratio

    Relative risk / Rate Ratio

    OUTCOME

    EXPOSURE

    Risk of outcome for exposed 15/100 = .15

    Risk of outcome for those not exposed 20/100 = .20

    Absolute risk difference = 0.20 - 0.15 = 0.05

    Rate ratio = 0.15 / 0.20 = 0.75

    If RR

    <1, incidence of outcome is less in exposed group

    =1, incidence of outcome is same in both groups

    >1, incidence of outcome is greater in exposed group


    Relative risk vs absolute risk difference

    Relative risk vs Absolute risk difference


    Hazard ratio

    Hazard Ratio

    Number of events per time period

    Number of exposed persons at risk per time period

    ÷

    Number of events per time period

    Number of non exposed persons at risk per time period


    Overall survival for dlbl vs il l edwards

    Overall Survival for DLBL vs IL (L Edwards)

    HR2.5 (95% CI 1.2-5.2)

    BCLU, n=34


  • Login