Download
1 / 37
370 likes | 513 Views
Mathematical Foundations of Robustness and Qualitative Robustness in Bootstrapping Mohammed Nasser and Nor Aishah Hamzah Institute of Mathematical Sciences University of Malaya. Contents. Interplay between Statistics, Mathematics and Computer Science
E N D
Mathematical Foundations of Robustness and Qualitative Robustness in Bootstrapping Mohammed Nasser and Nor AishahHamzah Institute of Mathematical Sciences University of Malaya
Contents • Interplay between Statistics, Mathematics and Computer Science • Mathematical Foundations of Robustness • Qualitative Robustness in Estmation • Qualitative Robustness in Bootstrapping
Interplay Among Statistics Mathematics and Computer Science • During the first three decades of the twentieth century three great revolutions swept across mathematics and science--- • the developmentof Quantum Mechanics (Heisenberg, 1930) after invention of quanta by Planck in 1900 and that of Relativity Theory ( Einstein, 1905 and 1916) in physics, • ii) axiomatization of Analysis and Algebra (, 1906 ; Hausdorff, 1914 etc ) following the success of Set Theory of Cantor (1874 and 1892), and of Non-Euclidean Geometries, and • iii) the beginning of Parametric Statistics (K. Pearson, 1900; Fisher, 1922 and 1925 etc).
Interplay Among Statistics Mathematics and Computer Science 1. They used Classical Analysis, Euclidean Geometry and Algebra. Their use of mathematics is not rigorous.They used both sample space X and parametric space W as subset of Rk, k, finite. Computation power was limited. K.Pearson (1900), Student (1908), R.A. Fisher (1922, 1925), J. Neyman and E.S. Pearson (1933) etc founded modern parametric inference. Cont.
Interplay Among Statistics Mathematics and Computer Science 2. Kolmogorov (1933), Cramer (1946), Wald (1947, 1950), Halmos and Savage(1949), von-Mises (1937 and 1947), Lehmann (1959) etc generalized the previous results and made theoretical statistics rigorous. They not only used Modern Analysis, Measure Theory, Point-set Topology and Functional Analysis but also developed them. X becomes topological spaces (mainly topological vector spaces other than Rk or their some subsets) Cont.
Interplay Among Statistics Mathematics and Computer Science 3. Both Non-parametric Statistics (Wilcoxon, 1945; Hoeffding, 1948; Hajek, 1969 etc) and Robust Statistics (Huber ,1964; Hampel,1968) are reactions to strict assumptions in Classical Parametric Inference and its inability in considering departures from the set of assumptions. Mathematical tools are both heuristic and rigorous. Computer intensive-techniques as well as rigorous functional analytic tools become indispensable for statistics. Wno longer remains within domain of Rk. Cont.
Interplay Among Statistics Mathematics and Computer Science 4. Though Rao (1945) and Jeffrey (1946) first introduced Classical DG (Differential Geometry) in Parametric Statistics to show relation between Riemannian Metric and Fisher Information, interest in Modern DG flared after the work of Efron (1975) have been steadily rising since eighties and nineties. Theoretical statistics used all the important concepts MDG. Also “New geometrical concepts were introduced” (Amari,1985). MDG is yet to be applied in Non-parametric and Robust Statistics. Meaningful works have been undertaken with semiparametric models (Bickel et al., 1993). Cont.
Interplay Among Statistics Mathematics and Computer Science • Recently Computer plays an important role in statistics. • Calculating fast classical statistical procedures and representing their summaries • Simulating Sampling Distributions specially when it is not analytically calculable or, valid approximation is not possible. • Bootstrapping (Weak simulation), Bagging (Bootstrap Aggregating), Boosting, MCMC and Data mining.
Sample QRI, SQRI(θ^) • =1/(1+n-1∑|(θ^[i]-θ^)|) • It’s maximum value is 1 • It’s minimum value is zero • The more SQRI is the more qualitative robust the estimator is • We can have plots to demonstrate sample QRI
Hampel in His Ph.D. Thesis (1968) Developed Three Concepts • Qualitative robustness ( also Π-robustness) • Breakdown point • Influence function • To assess robustness in estimation and thus raised rigorousness in robust estimation to a satisfactory level
Hampel (1968) Developed • Qualitative robustness to uphold qualitative side of robustness gauging distributional robustness. • Π-robustness– a form of qualitative robustness suitable for dependent observations. • Breakdown point to quantify global side of robustness. • Influence function to quantify infinitesimal side of robustness. Cont.
Hampel discussed and elaborated this concept at the outset of his thesis, breakdown point and influence function in the latter part. • His seminal article on qualitative robustness (1971) was published before his mostly quoted article on influence function (1974). • Breakdown point attracted wide range of researchers only after the development of finite version of breakdown point by Donoho (1982) and Donoho and Huber (1983). Cont.
Qualitative robustness has gained less popularity than other two concepts– influence function and breakdown point. • Why?? • It is no less important than the other two from the viewpoint of robustness. • Most probably its mathematical complic- ations and absence of finite versions have acted behind this present unpopularity.
Development After Hampel The concepts of qualitative robustness and Π robustness (more restrictive concept than qualitative robustness) were extended in different directions in last eighties. • Huber(1977,1981) modified Hampel's definition suggesting asymptotic equicontinuity of sampling distribution of the estimators with respect to n on the ground that nonrobustness gets worse for large n. • Rieder (1982) and Lambert (1982) introduced qualitative robustness in hypothesis testing, Cont.
Development After Hampel • Boente et al. (1987) following Papatoni-Kazakos and Gray (1979) and Cox (1981) generalized qualitative robustness for stochastic processes • Cuevas (1987 and 1988) adjusted some results of Hampel (1971) and Huber (1981) in the context of abstract inference.He showed incompatibility of consistency and qualitative robustness in the case of kernel density estimators Cont.
After Eighties We Came Across Few Works in These Area: • Cuevas and Romo (1993) and Nasser (2000) applied this concept in nonparametric bootstrapping and Basu et al. (1998) in Bayesian inference. • Daouia and Ruiz-Gazen (2004) etc studied qualitative robustness of nonparametric frontier estimator.
Let Tn:X Rk be an estimator where sample space is a polish space and parametric space, an open subset of Euclidean space. It induces a probability distribution on for every probability measure , the set of all probability measures on X. This can be shown as a mapping, hn between Hampel's Definition Cont.
SP() Sp(X) LF(Tn) LG(Tn) hn F G d2 d1 <ε <δ According to Hampel hn is equicontinuous at F for all n. Tnis robust at F. Huber argued that hnshould be asymptotically equicontinuous at F. Cont.
Meaningful Topology for Equicontinuity • Both argued that Prokhorov metric meet the demand of robustness. • It catches both gross error and rounding error, and also weak convergence. • Huber argued that when X or is R or subset of R, any metric like Levy that generates weak topology could be used for our purpose.
Hampel’s Results He deduced two main theorems, three lemmas and two corollaries to show the relation between concept of qualitative robustness and continuity of Tnin two cases • The general case Tn=Tn(Fn) and • 2) Particular case Tn=T(Fn) Cont.
Comment. Let Tn= T( Fn) be robust at F and consistent at all G in a nbd of F (with T∞(G)=T(G)). Then T is continuous at F.
Hampel’s Main Result Vs Huber’s Huber’s main result (Proposition 6.2 in his book) Assume that Tn (T(Fn)) is consistent in a nbd of F. Then T is continuous at F Tn is robust at F. Hampel’s FinalResult For Tn=T(Fn), T is continuous at all F Tn is robust and consistent,tending to T(F) at all F
All the results of Hampel and Huber can be generalized to the case of “generalized statistics” (statistics which take values in the general complete separable metric spaces"). It requires only two modifications; 1) using the metric d(x,y) of parametric space in place of |x-y| and adjusting the definitions with the metric. 2) applying Cantor’s Intersection Theorem for general complete metric space in proving lemma 2 in Hampel (1971)
