Nonparametric Methods I Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University firstname.lastname@example.org http://tigpbp.iis.sinica.edu.tw/courses.htm
Parametric vs. Nonparametric • MLE: probability distribution and likelihood • Bayes: conditional, prior and posterior distributions • Distribution free? • http://en.wikipedia.org/wiki/Non-parametric_statistics
Motivation (1) In many applications, direct access to a measurement and is not possible. However, an estimation of the measurement is needed. Most of the time, the large scale repetition of an experiment is not economically feasible. What can one do?
Motivation (2) • Q1: What estimator for the problem of interest can be used? • Q2: Having chosen an estimator, how accurate is it? What is the bias and variance of an estimator? • Q3: How to make inference? What is the confidence interval? What is the p-value for a hypothesis testing?
References • B. Efron (1979) Computers and the theory of statistics: thinking the unthinkable, SIAM Review, 21, 460-480. • B. Efron and R. J. Tibshirani (1993) An Introduction to the Bootstrap. Chapman & Hall. • J. I. De la Rosa and G. A. Fleury (2006) Bootstrap methods for a measurement estimation problem. IEEE Transactions onInstrumentation and Measurement, 55, 3, 820–827. • http://en.wikipedia.org/wiki/Resampling_%28statistics%29#Jackknifehttp://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29
Resampling Techniques • Data resampling • PART 1: Jackknife • Resampling without replacement • PART 2: Bootstrap • Resampling with replacement
PART 1: Jackknife • Naming • Illustration • Math Expression • Examples • R codes • C codes
Why the funny name of Jackknife? Jackknife: a pocket knife http://en.wikipedia.org/wiki/Jackknife Mosteller and Tukey (1977, p. 133) described a predecessor resampling method, the jackknife, in the following way: “The name ‘jackknife’ is intended to suggest the broad usefulness of a technique as a substitute for specialized tools that may not be available, just as the Boy Scout’s trustworthy tool serves so variedly…” http://mrw.interscience.wiley.com/emrw/9780470013199/esbs/article/bsa321/current/abstract
Illustration of Jackknife Population, Estimate by sampling inference resampling N times statistics
Jackknife by R 1. Open “R”
3.Select a mirror site, like Taiwan (Taipeh)
If you want to see the manual, you can type “?jackniffe”.
(1) Use mouse to select the R commands you want to run. (2) Press “F5” to run
Jackknife by C define functions
PART 2: Bootstrap • Naming • Illustration • Math Expression • Examples • R codes • Three approaches • Package(bootstrap) • Package(boot) • Write your own R codes • C codes
The Bootstrap • Bootstrap technique was proposed by Bradley Efron (1979, 1981, 1982) in literature. • Bootstrapping is an application of intensive computing to traditional inferential methods.
Why the funny name of bootstrap? • Bootstrap: http://www.concurringopinions.com/archives/Bootstrap_1.jpg • In the book of ‘Singular Travels, Campaigns and Adventures of Baron Munchausen’ by R. E. Raspe (1786), the main character, finding himself in a deep hole, extracts himself using only the straps of his boots. • http://tigger.uic.edu/~slsclove/stathumr.htm
Illustration of Bootstrap Population, estimate by sampling inference resampling B times statistics
Population step1 sampling
step2 resampling B times
Step 3: statistics
Bootstrap by R • Approach 1 • Use package “bootstrap” • Approach 2 • Use package “boot” • Approach 3 • Write your own R codes
Approach 1 http://finzi.psych.upenn.edu/R/library/bootstrap/DESCRIPTION