Boosting. LING 572 Fei Xia 02/02/06. Outline. Boosting: basic concepts and AdaBoost Case study: POS tagging Parsing. Basic concepts and AdaBoost. Overview of boosting. Introduced by Schapire and Freund in 1990s. “Boosting”: convert a weak learning algorithm into a strong one.
At round t, example xi has the weight Dt(i).
Problem #1 of Hw3
Error rate on a set of 27 benchmark problems
Training error is defined to be
#4 in Hw3: prove that training error
Training error drops exponentially fast.
T: the number of rounds of boosting
m: the size of the sample
d: VC-dimension of the base classifier space
(Problems #2 and #3 of Hw3)
“Gentle AdaBoost”, “BrownBoost”
h(x) = p1 if Φ(x) is true, h(x)=p0 o.w.
choose ht that minimizes Zt.
Choose the one with min Zt.
(x, 1), (x, 2), …, (x, k)