1 / 20

DATA ANALYSIS

DATA ANALYSIS. Module Code: CA660 Lecture Block 7:Non-parametrics. WHAT ABOUT NON-PARAMETRICS ? How Useful are ‘Small Data’?. General points - No clear theoretical probability distribution, so empirical distributions needed

biana
Download Presentation

DATA ANALYSIS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DATA ANALYSIS Module Code: CA660 Lecture Block 7:Non-parametrics

  2. WHAT ABOUT NON-PARAMETRICS? How Useful are ‘Small Data’? • General points • -No clear theoretical probability distribution, so empiricaldistributions needed • -So, less knowledgeof formof data* e.g. ranks instead of values • - Quick and dirty • - Need notfocus on parameter estimation or testing; when do - • frequently based on “less-good”parameters/ estimators, e.g. Medians; otherwise test “properties”, e.g. randomness, symmetry, quality etc. • - weaker assumptions, implicit in * • - smaller sample sizes ‘typical’ • - different data - implicit from other points. Levels of Measurement- Nominal, Ordinal typical for non-parametric/ distribution-free

  3. ADVANTAGES/DISADVANTAGES Advantages - Power may be better using N-P, if assumptions weaker - Smaller samples and less work etc. – as stated Disadvantages- alsoimplicit from earlier points, specifically: - loss of information /power etc. whendo know moreon data /when assumptionsdo apply - Separate tables each test General bases/principles: Binomial - cumulative tables, Ordinal data, Normal - large samples, Kolmogorov-Smirnov for Empirical Distributions - shift in Median/Shape, Confidence Intervals- more work to establish. Use Confidence Regions and Tolerance Intervals Errors– Type I, Type II . Power as usual. Relative Efficiency – asymptotic, e.g. look at ratio of sample sizes needed to achieve same power

  4. STARTING SIMPLY: - THE ‘SIGN TEST’ • Example.Suppose want to test if weights of a certain item likely to be more or less than 220 g. • From 12 measurements, selected at random, count how many above, how many below. Obtain 9(+), 3(-) • Null Hypothesis: H0: Median  = 220. “Test” on basis of counts of signs. • Binomialsituation, n=12, p=0.5. • For this distribution • P{3 X 9} = 0.962 while P{X 2 or X  10}= 1-0.962 = 0.038 • Result not strongly significant. • Notes:Need not be Median as “Location of test” • (Describe distributions by Location, dispersion, shape). Location = median, “quartile” or other percentile. • Many variants of Sign Test- including e.g. runs of + and - signs for “randomness”

  5. PERMUTATION/RANDOMIZATION TESTS • Example:Suppose have 8 subjects, 4 to be selected at random for new training. All 8 rankedin order of level of abilityafter a given period, ranking from 1 (best) to 8 (worse). • P{subjects ranked 1,2,3,4 took new training} = ?? • Clearly any 4 subjects could be chosen. Select r=4units from n = 8, • If new scheme ineffective, sets of ranks equally likely: P{1,2,3,4} = 1/70 • More formally, Sumranks in each grouping. Low sumsindicate that the trainingis effective, High sumsthat it is not. • Sums 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 • No. 1 1 2 3 5 5 7 7 8 7 7 5 5 4 2 1 1 • Critical Regionsize 2/70 given by rank sums 10 and 11 while • size 4/70 from rank sums 10, 11, 12 (both “Nominal”5%) • Testing H0: new training scheme no improvement vsH1: some improvement

  6. MORE INFORMATION WILCOXON ‘SIGNED RANK’ • Direction and Magnitude : H0:  = 220 ?Symmetry • Arrange all sample deviations from median in order of magnitude and replace by ranks (1 = smallest deviation, n largest). High sum for positive (or negative) ranks, relative to the other  H0 unlikely. • Weights 126 142 156 228 245 246 370 419 433 454 478 503 • Diffs. -94 -78 -64 8 25 26 150 199 213 234 258 283 • Rearrange 8 25 26 -64 -78 -94 150 199 213 234 258 383 • Signedranks 1 2 3 -4 -5 -6 7 8 9 10 11 12 • Clearly Snegative = 15 and < Spositive • Tables of form: Reject H0 if lower of Snegative, Spositive tabled value • e.g. here, n=12 at  = 5 % level, tabled value =13, so do notreject H0

  7. LARGE SAMPLES andC.I. • Normal Approximation for S the smaller in magnitude of rank sums • so C.I. as usual • General for C.I. Basic idea is to take pairs of observations, calculate mean and omit largest / smallest of (1/2)(n)(n+1) pairs. Usually, computer-based - re-samplingor graphicaltechniques. • Alternative Forms -common for non-parametrics • e.g. for Wilcoxon Signed Ranks. Use = magnitude of differences between positive /negative rank sums. Different Table • Ties- complicate distributions and significance. Assign mid-ranks

  8. KOLMOGOROV-SMIRNOV and EMPIRICAL DISTRIBUTIONS • Purpose- to compare set of measurements (two groups with each other) or one group with expected - to analyse differences. • CannotassumeNormalityof underlying distribution, (usual shape), so need enough sample values to base comparison on (e.g.  4, 2 groups) • Major features- sensitivity to differences in both shape and location of Medians: (does not distinguish whichis different) • Empirical c.d.f.not p.d.f. - looks for consistencyby comparing popn. curve (expected case) with empiricalcurve (sample values) • Step fn. • i.e. value at each step from data • S(x) should never be too far from F(x) = “expected” form • Test Basisis

  9. Criticisms/Comparison K-S with other ( 2) Goodness of Fit Tests for distributions Main Criticismof Kolmogorov-Smirnov: - wastes information in using only differences of greatest magnitude; (in cumulative form) General Advantages/DisadvantagesK-S - easy to apply - relatively easy to obtain C.I. - generally deals well with continuous data. Discrete data also possible, but test criteria not exact, so can be inefficient. - For two groups, need same number of observations - distinctionbetween location/shape differences not established Note: 2 applies to both discrete and continuous data , and to grouped, but “arbitrary” grouping can be a problem. Affects sensitivity of H0 rejection.

  10. COMPARISON 2 INDEPENDENT SAMPLES: WILCOXON-MANN-WHITNEY • Parallelwith parametric (classical)again. H0 : Samples from same population (Medians same) vs H1 : Medians not the same • For two samples, size m, n, calculate joint ranking and Sum for eachsample, giving Sm and Sn . Should be similarif populations sampled are alsosimilar. • Sm+Sn = sum of all ranks = and result tabulated for • Clearly, so need only calculate one from 1st principles • Tables typicallygive, for various m, n, the value to exceedfor smallest U in order to reject H0 . 1-tailed/2-tailed. • Easier : use the sum of smallerranks or fewer values. • Example in brief: If sum of ranks =12 say, probability based on no. possible ways of obtaining a 12 out of Total no. of possible sums

  11. Example - W-M-W • For example on weights earlier. Assume now have 2ndsample set also: • 29 39 60 78 82 112 125 170 192 224 263 275 276 286 369 756 • Combined ranksfor the two samples are: • Value 29 39 60 78 82 112 125 126 142 156 170 192 224 228 245 • Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 • Value 246 263 275 276 286 369 370 419 433 454 478 503 756 • Rank 16 17 18 19 20 21 22 23 24 25 26 27 28 • Here m = 16, n=12 and Sm= 1+ 2+ 3 + ….+21+ 28= 187 • So Um=51, and Un=141. (Clearly, can check by calculating Un directly also) • For a 2-tailed test at 5%level,Um= 53 from tables and our value is less, i.e. more extreme, so rejectH0 . Medians are different here

  12. MANY SAMPLES - Kruskal-Wallis • Direct extensionof W-M-W. Tests: H0: Medians are the same. • Rank total number of observations for all samples from smallest (rank 1) to highest (rank N) for N values. Ties given mid-rank. • rijis rank of observationxijand si = sum of ranks in ith sample (group) • Compute treatment and total SSQ ranks - uncorrectedgiven as • For no ties,this simplifies • Subtract off correction for average for each, given by • Test Statistic • i.e. approx. 2for moderate/large N. Simplifies further if no ties.

  13. PAIRING/RANDOMIZED BLOCKS - Friedman • Blocks of units, so e.g. twotreatments allocated at random within block = matched pairs; can use a variant of sign test(on differences) • Many samplesorunits= Friedman(simplest case of R.B. design) • Recallcomparisons withinpairs/blocks more precise than between, so including Blocks term, “removes” block effect as source of variation. • Friedman’s test- replaces observations by ranks (within blocks) to achieve this. (Thus, ranked data can also be used directly). • Have xij = response. Treatmenti, (i=1,2..t) in eachblock j, (j=1,2...b) • Ranked within blocks • Sum of ranks obtained each treatmentsi, i=1,…t • For rank rij(or mid-rank if tied), raw (uncorrected) rank SSQ

  14. Friedman contd. • With no ties, the analysis simplifies • Need alsoSSQ(All treatments –appear in blocks) • Again, the correction factor analogous to that for Kruskal-Wallis • and common form of Friedman Test Statistic • t, b not verysmall, otherwise need exacttables.

  15. Other Parallels with Parametric cases • Correlation- Spearman’s Rho( Pearson’s P-Mcalculated using ranks or mid-ranks) • where • used to compare e.g. ranks on two assessments/tests. • Regression– LSE robustin general. Some use of “median methods”, such as Theil’s (not dealt with here, so assume usual least squares form).

  16. NON-PARAMETRIC C.I. in Informatics: BOOTSTRAP • Bootstrapping = re-samplingtechniqueused to obtain Empirical distribution for estimator in construction of non-parametric C.I. • - Effective when distribution unknownor complex • - More computationthan parametric approaches and may fail when sample size of original experiment is small • - Re-samplingimplies sampling from a sample - usually to estimate empirical properties, (such as variance, distribution, C.I. of an estimator) and to obtain EDF of a test statistic- common methods are Bootstrap,Jacknife,shuffling • - Aim= approximate numerical solutions (like confidence regions). Can handle biasin this way - e.g. to find MLE of variance 2, mean unknown • - both Bootstrap and Jacknife used, Bootstrap more often for C.I.

  17. Bootstrap/Non-parametric C.I. contd. Basis- both Bootstrapand othersrely on fact that sample cumulative distn fn. (CDF or just DF) = MLE of a population Distribution Fn. F(x) DefineBootstrap sampleas a random sample,size n, drawn with replacementfrom a sample of n objects ForS the original sample, P{drawing each item, object or group} = 1/n Bootstrap sampleSBobtained from original, s.t. sampling n times with replacement gives Powerrelies on the fact that large number of resampling samples can be obtained from a single original sample, so if repeat process b times, obtain SjB, j=1,2,….b, with each of these being a bootstrap replication

  18. Contd. • Estimator- obtained from each sample. If is the estimate for the jthreplication, then bootstrap mean and variance • while BiasB= • CDFofEstimator = for b replications • soC.I. with confidence coefficient for some percentile is then • Normal Approx. for mean: Large b • (tb-1 - distribution if No. bootstrap replications small). Standardised Normal Deviate

  19. Example • Recall (gene and marker) or (sex and purchasing) example • MLE, 1000 bootstrapping replications might give results: • ParametricVariance 0.0001357 0.00099 • 95% C.I. (0, 0.0455) (0.162, 0.286) • 95% Interval (Likelihood) (0.06, 0.056) (0.17, 0.288) • BootstrapVariance 0.0001666 0.0009025 • Bias 0.0000800 0.0020600 • 95% C.I. Normal (0, 0.048) (0.1675, 0.2853) • 95% C.I. (Percentile) (0, 0.054) (0.1815, 0.2826)

  20. SUMMARY: Non-Parametric Use Simple property : Sign Tests – large number of variants. Simple basis Paired data Wilcoxon Signed Rank -Compare medians Conditions/Assumptions - No. pairs  6; Distributions- Same shape Independent data Mann-Whitney U - Compare medians - 2 independent groups Conditions/Assumptions -(N  4);Distributions same shape Correlation –as before. Parallels parametric case. Distributions: Kolmogorov-Smirnov - Compare either location, shape Conditions/Assumptions - (N  4), but the two are not separately distinguished. If 2 groups compared) need equal no. observations Many Group/Sample Comparisons: Friedman– compare Medians. Conditions : Data in Randomised Block design. Distributions same shape. Kruskal-Wallis- Independent groups. Conditions : Completely randomised. Groups can be unequal nos. Distributions same shape. Regression – robust, as noted, so use parametric form.

More Related