Incrementally Learning Parameter of Stochastic CFG using Summary Stats. Written by:Brent Heeringa Tim Oates. Goals:. To learn the syntax of utterances Approach : SCFG (Stochastic Context Free Grammar) M=<V,E,R,S> V-finite set of non-terminal E-finite set of terminals
Incrementally Learning Parameter of Stochastic CFG using Summary Stats
Written by:Brent Heeringa
V-finite set of non-terminal
E-finite set of terminals
R-finite set of rules, each r has p(r).
Sum of p(r) of the same left-hand side = 1
1)Expensive storage: need to store a corpus of complete sentences
2)Time-consuming: algorithms needs to repeat passes throughout all data
General method: Inside/Outside algorithm
Find expectation of rules
Maximize the likelihood given both expectation & corpus
Disadvantage of Inside/Outside algo.
Entire sentence corpus must be stored using some representation(eg. chart parse)
Expensive storage (unrealistic for human agent!)
(eg pt+1(r)=0.01* p t+1(r)
Need to store complete sentence corpus
Memory requirements is constant!