A semi theoretician s mid day confession the true meaning of i i d in applied statistics l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 56

A Semi-Theoretician's Mid-Day Confession : The True Meaning of i.i.d. in (Applied) Statistics PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

A Semi-Theoretician's Mid-Day Confession : The True Meaning of i.i.d. in (Applied) Statistics. Xiao-Li Meng Department of Statistics Harvard University. Crying and/or Smiling?. Size & Complexity of data are increasing ; Depth & Specificity of investigation goals are increasing ;

Download Presentation

A Semi-Theoretician's Mid-Day Confession : The True Meaning of i.i.d. in (Applied) Statistics

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A semi theoretician s mid day confession the true meaning of i i d in applied statistics l.jpg

A Semi-Theoretician's Mid-Day Confession:The True Meaning of i.i.d. in (Applied) Statistics

Xiao-Li Meng

Department of Statistics

Harvard University


Crying and or smiling l.jpg

Crying and/or Smiling?

  • Size & Complexity of data are increasing;

  • Depth & Specificity of investigation goals are increasing;

  • Available Time for conducting analysis is decreasing.

  • Grand Challenges = Great Opportunities


My two tales of nlaas l.jpg

My Two Tales of NLAAS

  • Statistics and Lies: How to use Bayesian modeling and multiple imputation to reduce response bias.

  • Disparities in Defining Disparity: Statistical conceptual frameworks (and theimpossibility of estimating disparity without making strong causal assumptions).


Overview of nlaas l.jpg

Overview of NLAAS

National Latino and Asian American Study (NLAAS)

  • NLAAS, conducted in 2002-2003, and made public on July 2007, is a national psychiatric epidemiologic study conducted to measure psychiatric disorders and mental health service usage in a nationally representative household sample of Asians and Latinos.

  • There are more than 5,000 variables (and the number is still growing!)

  • Total sample size is 4864: 2554 Latinos + 2095 Asians + 215 Whites


A huge questionnaire l.jpg

A HUGE Questionnaire!


Statistics can lie but can also correct for lies reducing response bias via bayesian imputation l.jpg

Statistics Can Lie But Can Also Correct For Lies: Reducing Response Bias Via Bayesian Imputation

Joint work with

Jingchen Liu1, Chih-nan Chen2, and Margarita Alegria2,3

1 Harvard University

2 Cambridge Health Alliance

3Harvard Medical School


Slide7 l.jpg

  • *SR2. Have you ever in your lifetime been admitted for an overnight stay in a hospital or other facility to receive help for problems with your emotions, nerves, mental health, or your use of alcohol or drugs?

    • YES1

    • NO5 GO TO *SR9.01

    • DON’T KNOW8GO TO *SR9.01

    • REFUSED9GO TO *SR9.01

  • *SR5a.Was this in the past month, past six months, past year, or more than a year ago?

    • PAST MONTH1GO TO *SR5c

    • PAST SIX MONTHS2GO TO *SR5c

    • PAST YEAR3GO TO *SR5c

    • MORE THAN A YEAR AGO4

    • DON’T KNOW8

    • REFUSED9

  • *SR5b.How old were you at the time of this admission?

    • _________ YEARS OLD

    • DON’T KNOW998

    • REFUSED999

  • *SR5c.How much time did you stay in the hospital during this admission?

    • ____________DURATION NUMBER

      75% people have this design


Slide8 l.jpg

  • *SR17. (IF *SR16 EQUALS ‘1’: Which ones? Just give me the letters. PROBE: Any other? / ALL OTHERS: (RB, PG 19) which of the following types of professionals did you ever see about problems with your emotions or nerves or your use of alcohol or drugs? Just give me the letters. (PROBE: Any others?) RECORD ALL MENTIONS

    • A. PSYCHIATRIST1

    • B. GENERAL PRACTITIONER OR FAMILY DOCTOR2

    • C. ANY OTHER MEDICAL DOCTOR, LIKE A CARDIOLOGIST OR

      (WOMEN: GYNECOLOGIST / MEN: UROLOGIST)3

    • D. PSYCHOLOGIST4

    • E. SOCIAL WORKER5

    • F. COUNSELOR6

    • G.ANY OTHER MENTAL HEALTH PROFESSIONAL, SUCH AS A PSYCHOTHERAPIST OR MENTAL HEALTH NURSE7

    • H.A NURSE, OCCUPATIONAL THERAPIST, OR OTHER HEALTH PROFESIONAL...……….8

    • I.A RELIGIOUS OR SPIRITUAL ADVISOR LIKE A MINISTER, PRIEST, PASTOR, RABBI9

    • J.ANY OTHER HEALER, LIKE AN HERBALIST, DOCTOR OF ORIENTAL MEDICINE, CHIROPRACTOR, SPIRITUALIST10

    • K.DON’T KNOW11

    • L.REFUSED12

      25% people have this design


Embedded experiment for detecting response bias l.jpg

Embedded Experiment for Detecting Response Bias

  • Interviewees were randomly divided into two groups. 75% received the standard ordering, 25% received the new ordering.

  • Why split 75-25, rather than 50-50?

  • How much of a difference does the ordering make?


Compare the old and new design service use l.jpg

Compare the Old and New Design – Service Use


Compare the old and new design other variables l.jpg

Compare the Old and New Design – Other Variables


Correcting reducing response biases via multiple imputation l.jpg

Correcting/Reducing Response Biases via Multiple Imputation

  • Goal: Correcting/reducing the underreporting of service use in the 75% group by using the data from the 25%.

  • A Grand Challenge: Imputation ≠ Randomization

    DESIRE: Imputed rates for 75% match the observed rates from 25% for any potential subpopulation of interest.

    REALITY: Can only include a handful of covariates due to identifiability, computational, and time constraints.

  • Why impute at all? Can’t we just use 25% for analysis?


Basic model setup l.jpg

Basic Model Setup

  • I: Group indicator: 0 for new design, and 1 for old

    design.

  • y: self-reported service use: 0 for no service, 1 for had service;

  • ξs: true service use: 0 for no service, 1 for had service;

  • ξl: lying under the old design: 0 for lying and 1 for telling the truth.

  • Of interest is the distribution of

    ξs| y=0, I=1.


  • First model for imputation l.jpg

    First Model For Imputation

    • Let’s assume

    • The likelihood function for one observation is

      where I=0 for 25% group, and I=1 for 75% group.


    Multivariate probit model l.jpg

    Multivariate Probit Model

    • We have 10 lifetime service use variables:

    • Associated with them, there are 10 lying indicators:

    • Introducing latent variable Z:

      is = I{zis > 0} and il = I{zil > 0}, where Z=(z1s,…, z10s, z1l,…,z10l)

      11 > 0 is a 10×10 with diagonal elements all 1, and

      where x is a p×1, and B 20×p.


    Covariates l.jpg

    Covariates

    • Categorical variables: marital status, insurance status, working status, region in the country, ethnicity, immigration status, gender, psychiatric disorder diagnostics.

    • “Continuous” variables: logarithm of annual income, total number of psychiatric disorders, social status, age, k10 distress (psychiatric disorder related variable).

    • Some categorical variables are treated as “continuous” to reduce “dummy variables”, hence the number of covariates.

    • These variables are “negotiated” with psychologists.


    Lifetime service use results l.jpg

    Lifetime Service Use Results


    Any service rates l.jpg

    Any Service Rates


    High order interactions l.jpg

    High Order Interactions

    • Two way interaction:

    • High order interactions:

    • Failure to capture high order interactions will lead to biases in imputing maxi Xi , hence biases for the “any rates”.

    • Though arguably this “over-imputation” might be correcting for the false correlation induced by the interaction between memory decay and lying behavior, because it was not observed for the “last-12 month” imputation.


    Model improvement l.jpg

    Model Improvement

    • The first model failed to capture the high-order interactions leading to the over-imputation of the “overall rates”.

    • Need hierarchical modeling using natural service type:

      Specialist, Generalist, Human Services, Alternative Services

    • We use  as the type indicator and  the service use within each type.


    Second model continuation ratio model l.jpg

    Second Model - Continuation Ratio Model

    • Let

      j1 and j2 follows multivariate probit model, that is,

    • z1={zj1} and z2={zj2} are independent and

    • A strong prior on  is imposed to avoid non-identifiability.

      b ~ N(2, 0.01),  ~ Inv-Wishart(I, 16)

    • When  →∞, this model reduces to the first model.

    • Cox (1972, JRSSB) proportional hazard model

      Heagerty and Zeger (2000, Biometrics) multivariate continuation ratio model.


    Lifetime service use results second model l.jpg

    Lifetime Service Use Results – Second Model


    Imputation quality checking stratified by variables included in the model l.jpg

    Imputation Quality Checking: stratified by variables included in the model


    Imputation quality checking stratified by variables not included in the model l.jpg

    Imputation Quality Checking: stratified by variables not included in the model


    Why the problem is so hard stratified by insurance type included in the model l.jpg

    Why the problem is so hard? – Stratified by insurance type (included in the model)


    Effective sample size l.jpg

    Effective sample size


    The impact of the weights l.jpg

    The impact of the weights


    The failing of randomization l.jpg

    The Failing of Randomization


    Endless future work l.jpg

    Endless Future Work

    • Understand the CR model better via model diagnosis;

    • Further model improvement for subpopulations;

    • More efficient MCMCs for general applications;

    • General model strategy with many variables, small sample sizes, and potentially many subpopulations of interest.

    • “Data mining” tools for discovering problematic subpopulations(almost the same as discovering “bad genes”), even just for “buyers be aware” .


    Disparities in defining disparity statistical conceptual frameworks l.jpg

    Disparities in Defining Disparity: Statistical Conceptual Frameworks

    • Joint work with

    • Naihua Duan1,2, Julia Lin3, Chih-nan Chen3, Margarita Alegria3,4

      1 University of California, Los Angeles

      2 Columbia University and New York State Psychiatric Institute

      3Cambridge Health Alliance

      4 Harvard Medical School


    Ambiguities in iom s definition l.jpg

    Ambiguities in IOM’s Definition

    • IOM (2003): Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare

    • IOM defines disparities in healthcare as “racial or ethnic differences in the quality of healthcare that are not due to access-related factors or clinical needs, preferences, and appropriateness of intervention.”

    • Explicit recognition of the role of causality.


    Important issues in iom s definition of healthcare disparity l.jpg

    Important Issues in IOM’s Definition of Healthcare Disparity

    • Difference is not necessarily disparity (which is not necessarily discrimination)

    • Justifiable (to be adjusted) vs. not justifiable (not to be adjusted)

      • Differences due to age, pre-existing health condition, etc., are OK

      • Difference due to literacy is not OK

    • Should access be adjusted for or not?

    • Grant Challenge: Causal relationship (“not due to”)

      Need to clearly spell out the causal relationships between “justifiable variables” and “not justifiable variables”


    Rubin causal model l.jpg

    Rubin Causal Model

    • Causal model based on counterfactuals

    • Thought experiment, or actual experiment

      • What is changed

      • What is left unchanged

    • There is no single right or wrong thought experiment; different thought experiments should be developed for different clinical or policy goals


    Statistical conceptual frameworks l.jpg

    Statistical Conceptual Frameworks

    • Conceptual framework for conditional disparity, marginal disparity, and joint disparity

    • Define estimand: what do we want to estimate?

    • Estimation methods are important only after we understand what they intend to estimate.


    Data setting l.jpg

    Data Setting

    • Univariate Outcome Y: service use; number of visits.

    • Covariates for Adjustment X(A) : disorders; age; etc.

    • Covariates not for Adjustment X(N) : income; education; etc.

    • Deciding what to adjust is usually not a statistician’s task (though we can help), but how to adjust is.


    Always thinking jointly l.jpg

    Always Thinking Jointly …

    • Observed Data:

    • Key distributions: race-specific joint distribution.

    Race R, Y, X(A), X(N); and Weights W

    P(Y, X(A), X(N) | R)

    =P[Y|X(A), X(N), R]P[X(N)|X(A), R]P[X(A)|R]

    =P[Y|X(A), X(N), R]P[X(A)|X(N), R]P[X(N)|R]


    Common unrealizable disparity l.jpg

    Common “Unrealizable” Disparity


    So what happens to the right eye l.jpg

    So what happens to the right eye?

    • Vision Acuity (AV) of left and right eyes are positively correlated in the population.

    • Suppose laser surgery is done to correct the AV of the left eye (L). What is going to happen to the AV of the right eye (R)?

    • We want to model P(R|L, S), where S=Surgery.

    • Will P(R|L, S)=P(R|L)? (e.g., L “causing” R)

    • Will P(R|L, S)=P(R)? (e.g., R “causing” L)

    • Orwill P(R|L, S) be something not estimable from the pre-surgery population? (and cross-sectional data …)


    Conditional disparity l.jpg

    Conditional Disparity


    Conditional disparity assumed causal model l.jpg

    Conditional Disparity: Assumed Causal Model

    X(N)

    Y

    R

    X(A)

    X(N): Not for adjust

    X(A): Adjust


    Conditional disparity thought experiment l.jpg

    Conditional Disparity: Thought Experiment

    X(N)

    Y

    R

    X(A)


    Marginal disparity l.jpg

    Marginal Disparity


    Marginal disparity50 l.jpg

    Marginal Disparity

    X(N)

    Y

    R

    X(A)


    Marginal disparity51 l.jpg

    Marginal Disparity

    X(N)

    Y

    R

    X(A)


    Joint disparity l.jpg

    Joint Disparity

    X(N)

    Q

    R

    X(A)


    Possible strategies for joint disparity l.jpg

    Possible Strategies for Joint Disparity

    • Joint disparity perhaps is “sandwiched” between conditional disparity and marginal disparity???

    • Consider the joint disparity causal model as a mixture of the conditional disparity model and the marginal disparity model.

    • Will need assumptions on human behavior.

    • Further look into literatures such as Economics, Epidemiology – this cannot be (entirely) new!


    Meeting challenges a paradigm shift l.jpg

    Meeting Challenges: A Paradigm Shift?

    • Traditionally we go from ad hoc methods to principled methods

    • Need a lot more Principled Quick and Dirty (PQD) methods

    • Teach and research more about setting tolerable errors and when and how tocut corners

    • Need a Comparative Evaluation Framework (i.e., which of the two is a better method given the problem at hand), in contrast to the Absolute Evaluation Framework (e.g., asymptotic consistency; most efficient estimator)


    So what is i i d l.jpg

    So what is “i.i.d”?

    • Google shows 17,000,000 items for “i.i.d”

    • “independent and identically-distributed” made No. 2!

    • No. 1 is “Imperial Irrigation District” because its URL is www.iid.com

    • My favorite: “Idea I Do” (No idea what does this mean!)

    • Joe’s very clever answer: “It is difficult”!

    • Your answer?


    The obvious answer l.jpg

    The Obvious Answer :

    • Doing good (applied) statistics is

      “inherently and increasingly difficult”

      (The first sentence of my abstract!)


  • Login