Part-of-Speech Tagging - PowerPoint PPT Presentation

part of speech tagging n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Part-of-Speech Tagging PowerPoint Presentation
Download Presentation
Part-of-Speech Tagging

play fullscreen
1 / 27
Part-of-Speech Tagging
131 Views
Download Presentation
jaguar
Download Presentation

Part-of-Speech Tagging

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Part-of-Speech Tagging 인공지능 연구실 정 성 원

  2. The beginning • The task of labeling (or tagging) each word in a sentence with its appropriate part of speech. • The representative put chairs on the table AT NN VBD NNS IN AT NN • Using Brown/Penn tag sets • A problem of limited scope • Instead of constructing a complete parse • Fix the syntactic categories of the word in a sentence • Tagging is a limited but useful application. • Information extraction • Question and answering • Shallow parsing

  3. The Information Sources in Tagging • Syntagmatic: look at the tags assigned to nearby words; some combinations are highly likely while others are highly unlikely or impossible • ex) a new play • AT JJ NN • AT JJ VBP • Lexical : look at the word itself. (90% accuracy just by picking the most likely tag for each word) • Verb is more likely to be a noun than a verb

  4. Notation • wi the word at position i in the corpus • ti the tag of wi • wi,i+m the words occurring at positions i through i+m • ti,i+m the tags ti … ti+m for wi … wi+m • wl the lth word in the lexicon • tj the jth tag in the tag set • C(wl) the number of occurrences of wl in the training set • C(tj) the number of occurrences of tj in the training set • C(tj,tk) the number of occurrences of tj followed by tk • C(wl,tj) the number of occurrences of wl that are tagged as tj • T number of tags in tag set • W number of words in the lexicon • n sentence length

  5. The Probabilistic Model (I) • The sequence of tags in a text as Markov chain. • A word’s tag only depends on the previous tag (Limited horizon) • Dependency does not change over time (Time invariance) • compact notation : Limited Horizon Property

  6. The Probabilistic Model (II) • Maximum likelihood estimate tag following

  7. The Probabilistic Model (III) (We define P(t1|t0)=1.0 to simplify our notation) • The final equation

  8. The Probabilistic Model (III) • Algorithm for training a Visible Markov Model Tagger Syntagmatic Probabilities: for all tags tjdo for all tags tkdo P(tk | tj)=C(tj, tk)/C(tj) end end Lexical Probabilities: for all tags tjdo for all words wldo P(wl | tj)=C(wl, tj)/C(tj) end end

  9. The Probabilistic Model (IV) <Idealized counts of some tag transitions in the Brown Corpus>

  10. The Probabilistic Model (V) <Idealized counts for the tags that some words occur with in the Brown Corpus>

  11. The Viterbi algorithm comment : Given: a sentence of length n comment : Initialization δ1(PERIOD) = 1.0 δ1(t) = 0.0 for t ≠ PERIOD comment : Induction for i := 1 to n step 1 do for all tags tj do δi+1(tj) := max1<=k<=T[δi(tk)*P(wi+1|tj)*P(tj|tk)] ψi+1(tj) := argmax1<=k<=T[δi(tk)*P(wi+1|tj)*P(tj|tk)] end end comment : Termination and path-readout Xn+1 = argmax1<=j<=Tδn+1(j) for j := n to 1 step – 1 do Xj = ψj+1(Xj+1) end P(X1 , … , Xn) = max1<=j<=Tδn+1(tj)

  12. Variations (I) • Unknown words • Unknown words are a major problem for taggers • The simplest model for unknown words • Assume that they can be of any part of speech • Use morphological information • Past tense form : words ending in –ed • Capitalized

  13. Variations (II) • Trigram taggers • The basic Markov Model tagger = bigram tagger • two tag memory • disambiguate more cases • Interpolation and variable memory • trigram tagger may make worse pridictions than a bigram tagger • linear interpolation • Variable Memory Markov Model

  14. Variations (III) • Smoothing • Reversibility • Markov model decodes from left to right = decodes from right to left Kl is the number of possible parts of speech of wl

  15. Variations (IV) • Maximum Likelihood: Sequence vs. tag by tag • Viterbi Alogorithm : maximize P(t1,n|w1,n) • Consider : maximize P(ti|w1,n) • for all i which amounts to summing over different tag sequance • ex) Time flies like a arrow. • a. NN VBZ RB AT NN. P(.) = 0.01 • b. NN NNS VB AT NN. P(.) = 0.01 • c. NN NNS RB AT NN. P(.) = 0.001 • d. NN VBZ VB AT NN. P(.) = 0 • one error does not affect the tagging of other words

  16. Applying HMMs to POS tagging(I) • If we have no training data, we can use a HMM to learn the regularities of tag sequences. • HMM consists of the following elements • a set of states ( = tags ) • an output alphabet ( words or classes of words ) • initial state probabilities • state transition probabilities • symbol emission probabilities

  17. Applying HMMs to POS tagging(II) • Jelinek’s method • bj.l : probability that word (or word class) l is emitted by tag j

  18. Applying HMMs to POS tagging(III) • Kupiec’s method |L| is the number of indices in L

  19. Transformation-Based Learning of Tags • Markov assumption are too crude→ transformation-based tagging • Exploit a wider range • An order of magnitude fewer decisions • Two key components • a specification of which ‘error-correcting’ transformations are admissible • The learning algorithm

  20. Transformation(I) • A triggering environment • A rewrite rule • Form t1→t2 : replace t1 by t2

  21. Transformation(II) • environments can be conditioned • combination of words and tags • Morphology-triggered transformation • ex) Replace NN by NNS if the unknown word’s suffix is -s

  22. The learning algorithm C0 := corpus with each word tagged with its most frequent tag for k:=0 step 1 do ν:=the transformation ui that minimizes E(ui(Ck)) if (E(Ck)-E(ν(Ck))) < Єthen break fi Ck+1:= ν(Ck) τk+1:= ν end Output sequence: τ1, …, τk

  23. Relation to other models • Decision trees • similarity with Transformation-based learning • a series of relableing • difference with Transformation-based learning • split at each node in a decision tree • different sequence of transformation for each node • Probabilistic models in general

  24. Automata • Transformation-based tagging has a rule component, it also has a quantitative component. • Once learning is complete, transformation-based tagging is purely symbolic • Transformation-based tagger can be converted into another symbolic object • Roche and Schobes(1995) : finite state transducer • Advantage : speed

  25. Other Method, Other Languages • Other approaches to tagging • In chapter 16 • Languages other than English • In many other languages, word order is much freer • The rich inflections of a word contribute more information about part of speech

  26. Tagging accuracy • 95%~97% when calculated over all words • Considerable factors • The amount of training data available • The tag set • The difference between training set and test set • Unknown words • a ‘dump’ tagger • Always chooses a word’s most frequent tag • Accuracy of about 90% • EngCG

  27. Applications of tagging • Benefit from syntactically disambiguated text • Partial Parsing • Finding none phrases of sentence • Information Extraction • Finding value for the predefined slots of a template • Finding good indexing term in information retrieval • Question Answering • Returning an appropriate noun such as a location, a person, or a date