BMS 617

1 / 30

# BMS 617 - PowerPoint PPT Presentation

BMS 617. Lecture 7 – T-tests. T-tests. T-tests refer to a family of statistical tests, in which a mean, or difference in two means, is assumed to be sampled from a T-distribution As a statistical test, T-tests compute p-values

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## BMS 617

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### BMS 617

Lecture 7 – T-tests

Marshall University Genomics Core Facility

T-tests
• T-tests refer to a family of statistical tests, in which a mean, or difference in two means, is assumed to be sampled from a T-distribution
• As a statistical test, T-tests compute p-values
• The probability of seeing a mean, or difference of means, this large, assuming the null hypothesis is true
• Interpreting a t-test involves knowing the null hypothesis

Marshall University School of Medicine

Types of T-test
• One-class T-test:
• Null hypothesis is that the mean of a set of values is equal to some fixed value
• Two-class T-tests:
• Two groups of values
• Unpaired T-test
• Most common T-test
• Null hypothesis is that the values in each group are sampled from distributions with equal means
• Paired T-test
• Each sample in the first group is paired with a sample in the second group
• Null hypothesis is that the mean of the difference between each pair is zero

Marshall University School of Medicine

Example: One-class T-test
• Recall our body-temperature data from earlier
• n=130 samples of body temperature
• Mean m=36.82C, SD s=0.41C
• We wanted to use this to test the hypothesis that mean body temperature was μ=37C
• Under the assumptions of the one-class t-test, the value t=(μ-m)/(s/√n) follows a t-distribution with n-1 degrees of freedom
• For this example, t=5.006

Marshall University School of Medicine

Computing a p-value for a given t
• To get a p-value for t=5.006, we either use software or a table
• Need to know the degrees of freedom
• In this case, d.f.=130-1=129
• Tables either give the probability value for a given t and df, or critical t-values for given probabilities and df
• From tables or software, the probability that t<=5.006 is p=0.9999991
• We want to know the probability of seeing a result this extreme, assuming the mean is 37C, i.e. assuming t follows a t-distribution with 129 d.f.
• This is the probability either that t>5.006 or that t<-5.006
• A two-tailed, one-class t-test
• P(t>5.006)=1-0.9999991=0.0000009
• So P(t<-5.006)=0.0000009
• and p=2 x 0.0000009 = 0.0000018

Marshall University School of Medicine

Assumptions for a one-class t-test
• The one-class t-test is accurate under the following assumptions:
• The samples are random (or representative)
• The observations are independent
• The data are accurate
• The data are sampled from a population that is normally distributed

Marshall University School of Medicine

One-class T-test and the Confidence Interval
• We saw earlier how to compute a confidence interval for the mean of this data set
• Calculate w=t*s/√n
• The confidence interval is from m-w to m+w
• t* is the value from the t-distribution for which P(t>t* or t<-t*)=1-confidence
• e.g. for a 95% confidence interval we want P(t>t* or t<-t*) =0.05
• So P(t>t*)=0.025. From tables or software, t*=1.979.
• This gives a 95% confidence interval of [36.75, 36.89]
• Knowing the 95% confidence interval does not contain the null hypothesis value of 37 is equivalent to knowing the p-value is less than 0.05

Marshall University School of Medicine

Two-class unpaired T-test: Example
• For an example of a two-class, unpaired T-test, consider the GRHL2 expression data we saw earlier from Cieply et al., Cancer Research 2012.
• Compared expression of GRHL2 in different breast cancer cell lines, classified as Basal A, Basal B, or Luminal.
• Compare the Basal A expression to the Basal B expression

Marshall University School of Medicine

GRHL2 Expression Data

Marshall University School of Medicine

GRHL2 Expression Data

Marshall University School of Medicine

How an unpaired t-test works

An unpaired t-test works by computing the difference of the means of the two samples

Assuming the null hypothesis – that the difference of the two means is zero – the difference of the sample means, divided by a pooled standard error of the mean, will be distributed with a t-distribution

Marshall University School of Medicine

Unpaired t-test for the GRHL2 data

Marshall University School of Medicine

Assumptions for the unpaired T-test
• The unpaired T-test works along essentially the same assumptions as the one-class T-test:
• The samples are random or representative
• The observations are independent
• The data are accurate
• The values in the populations are at least approximately normally distributed
• Additionally, the t-test we used here assumes:
• The the populations have the same standard deviation

Marshall University School of Medicine

Assumption of equal variances
• In the Basal A vs Basal B GRHL2 comparison, the Basal B samples have higher SD (0.7859) than the Basal A samples (0.4463)
• The t-test we ran assumed the samples came from populations with equal variances (i.e. equals standard deviations)
• A test can be run to see if the data are consistent with the assumption of equal variances
• The distribution of the square of the ratio of the standard deviations is known under the assumption that the population variances are equal

Marshall University School of Medicine

If the assumption of equal variances is violated
• A modified t-test can be used, which doesn’t make the assumption of equal variances
• Called the “Welch T-test”
• Has less power than the standard unpaired t-test
• As usual, testing your data set in order to decide which test to use can give misleading results
• Typically will give over-optimistic p-values
• In the ideal world, we would run an experiment specifically to determine if the assumption of equal variances holds, then use that to determine how to analyze our real experiment

Marshall University School of Medicine

Rules of thumb for the assumption of equal variances
• Unequal variances will only badly affect the t-test if the number of samples in each group is small and unequal
• In other cases the t-test is very robust to violations of this assumption
• In practice, I do the following:
• If the number of samples is equal, I use the regular t-test
• If the number of samples in both groups is at least 5, no matter if they are equal, I use the regular t-test
• If there is reason to believe the variances should be equal (e.g. if all the variance comes from technical replicates), I use the standard t-test
• Otherwise, I use the Welch T-test

Marshall University School of Medicine

95% Confidence Intervals and Unpaired t-tests

Marshall University School of Medicine

95% Confidence Intervals and unpaired T-tests
• The unpaired t-test results computed the difference between the means and the 95% confidence interval for that difference
• For this example the 95% confidence interval of the difference of the means was [-2.373, -1.348]
• If this confidence interval doesn’t contain zero, this is equivalent to p<0.05
• We can also compute the confidence interval for each mean independently
• If these confidence intervals do not overlap, then the p value is definitely less than 0.05
• In fact, it must be a lot less…
• If the confidence intervals do overlap, then p may or may not be less than 0.05
• Cannot deduce anything from the error bars in this case

Marshall University School of Medicine

Other error bars and statistical significance
• What if the bar chart uses SD or SEM for the error bars
• SD tells us about the amount of scatter in the data
• Nothing about the precision with which the mean is measured
• Overlapping, or non-overlapping, SD error bars have nothing to do with statistical significance
• SEM measures the precision with which we approximate the mean
• But interpretation depends on knowing the sample size
• We can deduce the following:
• If SEM error bars overlap, then the difference is definitely not statistically significant at p=0.05 (in fact, p is much bigger…)
• If SEM error bars do not overlap, the p may or may not be less than 0.05

Marshall University School of Medicine

Error bars and statistical significance summary

Marshall University School of Medicine

Paired t-tests
• Paired t-tests are used when the comparison is between samples which are paired between the groups
• Before and after treatments on a set of patients
• Pair the “before value” on patient A with the “after value” on patient A
• The “before value” on patient B with the “after value” on patient B, etc
• Studies in which subjects are recruited to two groups in a matched fashion
• Match a control patient with a treatment patient based on age, sex, weight, height…
• Difficult type of study to perform
• Twin or sibling studies
• Lab experiments in which treated and control samples are handled in parallel
• Plate cells, divide into two, treat one half and use the other as control
• Repeat the next day with another plate, etc

Marshall University School of Medicine

Example
• In a recent experiment, we performed expression profiling on a set of eight mice
• Should not use t-test here without correcting for multiple hypotheses, but this is a good example for demonstration
• Four litters of mice were bred, and two male mice selected (at random if necessary) from each litter
• One mouse from each pair was treated and one was used as a control
• Analyzing these data with a paired test has the potential to eliminate any litter-litter variation

Marshall University School of Medicine

Example data
• Actual data are read counts for the gene of interest
• After sequencing, align all reads to the genome
• Count the number of reads that align to each gene for each sample

Marshall University School of Medicine

Paired t-test
• There are two distinct null hypotheses we can make about paired data
• Both say “the data is no different between the two groups”
• One is that the difference between the values in the groups is zero (“paired t-test”)
• The other is that the ratio of the values in the groups is 1 (“ratio paired t-test”)

Marshall University School of Medicine

Paired T-test results

Marshall University School of Medicine

How a paired t-test works
• A paired t-test is really just a one-class t-test!
• Computes the differences for each pair
• And then tests the null hypothesis that those differences are samples from a normal distribution with mean zero
• The confidence interval is just the confidence interval of the mean differences

Marshall University School of Medicine

Ratio paired t-tests
• The ratio paired t-test tests the null hypothesis that the ratio of the paired values is 1
• This is done simply by a mathematical trick: take the log of all the ratios
• Then perform a regular paired t-test with the log ratios instead of the differences
• Log(1)=0
• Software performing a ratio paired t-test takes care of computing the logs, performing the test, and then transforming the mean difference of log ratios, confidence interval of that mean, etc, back to ratio values

Marshall University School of Medicine

Graphing paired data
• Plotting bar charts, or even column-scatter plots, of paired data does not show the pairing between the data values
• A better presentation is a connected column scatter plot
• Column scatter plot with lines connecting the paired data points

Marshall University School of Medicine

Connected column scatter plot

Marshall University School of Medicine

t-test summary
• t-tests are a family of statistical hypothesis tests
• Generate a p-value
• Remember how to interpret!
• Null hypotheses:
• For a one class t-test, the null hypothesis is that the samples are drawn from a normally-distributed population with a specified mean
• For an unpaired t-test, the null hypothesis is that the samples are drawn from two normally-distributed populations with equal means
• For a paired t-test, the null hypothesis is that the differences between matched values are samples of a normally-distributed population with mean zero
• Equivalent to a one class t-test on the differences between matched values

Marshall University School of Medicine