1 / 30

A Corpus Based Computational Linguistics

A Corpus Based Computational Linguistics. Alon Itai Department of Computer Science Technion—Israel Institute of Technology, Haifa, Israel. Introduction. Language is the communication media between humans If we want to allow computers to communicate with us,

dillan
Download Presentation

A Corpus Based Computational Linguistics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Corpus Based Computational Linguistics Alon Itai Department of Computer ScienceTechnion—Israel Institute of Technology, Haifa, Israel

  2. Introduction Language is the communication media between humans If we want to allow computers to communicate with us, we have to teach them human language (aka “Natural Language”) Investigating natural language is a complex and interesting challange It also has many immediate technological benefits, such as: Internet search engines voice interface with computers– for children, handicapped persons.

  3. Search Google for “dog” • Dogpile Web Search Home Pagedog.com - a dog's best friend • Sausage Software - HotDog Web Editors - HotDog Professional, ...... Special. Make certain you check out our Dog Packs before making yourpurchase. דפים דומים • explodingdog • Dog-Play: Great activities you can do with your dogActivities for Dogs. Having Fun with Your Dog. ...

  4. Search Google for “dogs” • I-Love-Dogs.com - Free Resources For Dog Lovers!including: Free Dog LoversI-Love-Dogs.com - Free Dog Resources For Dog Lovers including… • Dogs in CanadaCanada's Top Obedience Dogs 2002. • Guide Dogs for the BlindGuide Dogs for the Blind A nonprofit, charitable organization (800) 295-4050 • Dogs of the Dow - High dividend yield Dow stocks - www. ...In addition to the high dividend yield stocks of the Dogs of the Dow, you will also find a helpful set of long-term stock market charts, investment research

  5. Ambiguity—the Main Problem • Morphological בחור • Lexicalbank • SyntacticalI saw the man with the telescope • Semantichot dogs for the blind • Pragmatic: Can you pass the salt?

  6. Methods for understanding Language The traditional method Develop a theory, build tools,… Corpus methods

  7. Hebrew Computational Linguistics Most research is in English Many other languages have developed tools Why not in Hebrew ? Textkorpora des IDS The Oslo Corpus of Tagged Norwegian Texts Корпус от разговорен български език

  8. Special problems of Hebrew אבגדהוזחטיכלמנסעפצקרשת • The writing systemTechnical problems (right to left), font, different writing systems,no standard • Complex morphology"עזבתני" = You have left me4 morphemes in one word • Ambiguityהקפה = ה+קפה, הקפה, הקפהּ בחור = ב+ה+חור, בחוּר some words can have upto 13 different readings (שמונה)

  9. Achievements • Constructed an analyzer to read Hebrew text that is 96% correct. • Constructing data base for automatically constructing a syntactic analyzer of Hebrew • Search engines

  10. סיכום וחזון • נביא את המחקר בעברית למצבן של שפות אחרות • נלמד מעברית על שפות דומות • נפתח שיטות לטיפול בשפה כלשהי • נפתח מנגנונים להבנת טכסטים • נבין טוב יותר כיצד המוח האנושי פועל • לעבור את מבחן טיורינג .

  11. The Problem Written Hebrew texts are ambiguous. The reasons • The vowels and gemination are omitted. קופה = קוּפָּה קוׂפָה • small words are prepended. וכשתלך = ו + כש + תלך = and when you will go • Hebrew morphology is complex

  12. The structure of a Hebrew word morphemes • the lexical lemma, • short words such as determiners, prepositions and conjunctions prepended to the word, • suffixes for possessives and object clitics. • The linguistic features mark part-of-speech (POS), tense, person etc. linguistic features

  13. Example • $QMTI שקמתי • $iqmati – ֹשִקְמָתִי my sycamore • $e-qamti – ֹשֶקַמְתִי that I got up • $e-qammati – ֹשֶקַמָּתִי that my hey noun sg possessive-1sg connective+verb 1sg past connective + noun sg possessive-1sg

  14. Previous work

  15. POS and Morphological disambiguation • jhjh

  16. Three stages • Word stage – find the most probable reading of a word regardless of its context. • Pair stage –correct the analysis of a word based on the analysis of its immediate neighbors. • Sentence stage – use a syntactic parser to rule out improbable analyses. Combining all three stages yielded the best results

  17. The Word Stage • Give each word its most probable analysis. ☹ How to estimate the probability of each analysis? ☺ Estimate the probability of each analysis from a large analyzed corpus. ☹ A large enough corpus does not exist. ☹ Since each word has many forms, the number of word tokens is so large that many word forms won’t appear even in 10M word corpus.

  18. The Word Stage • Following the “Similar Words Method”,(Levinger, Ornan and Itai 1995) estimate the probability of each analysis of an ambiguous word by changing a (single) feature of each analysis, and comparing the occurrences of the resultant words in a large corpus. • Example: HQPHהקפה • the coffee: definite to indefinite QPH • encirclement: indefinite to definite HHQPH • her perimeter: feminine possessive to masculine possessive HQPW. • Distribution: QPH=180, HHQFH=18, HQPW=2 and the winner is הקפֶה (the coffee).

  19. Our variation of the SW method • To overcome sparseness, we assumed that the lemma and the other morphemes/linguistic features are statistically independent.Namely, P(the coffee) = P(the)P(coffee). • Even though the assumption is not valid, the resultant ranking is correct.

  20. Evaluation and Complexity • Errors 36%  14.5% • Complexity of algorithm O(n), where n is the size of the corpus. • Keeping a copy of the corpus as an inverse file reduces the complexity to linear in the number of different similar words.

  21. The pair stage • Following Brill, we learned correction rules from a corpus. • The initial morphological score of an analysis is its probability as obtained at the word stage. • Correction rules modify the scores by considering pairs of adjacent words, checking if the rule applies, and if so modify the scores.

  22. Example of a correction rule If the POS of the current tag of w1 is a proper-noun and the POS of the current tag of w2 is a noun and w2 has an analysis as a verb that matches w1 by gender and number, then add 0.5 to the morphological score of w2 as a verb, and normalize the scores .

  23. 0.467 0.8 0.533 Example YWSP &DR עדריוסף • YWSP=proper noun masc.(Joseph) • &DR =noun masc. sg. abs indef (herd) score=0.7 • &DR = verb past 3sg masc. (hoed) score=0.3 normalization

  24. Learning the Rules from a training corpus Input: A training corpus, where each word is correctly analyzed. • Run the word stage on the training corpus. • Generate all possible rules. • For each rule, set the correction factor to be the minimum value that does more good than damage. • Choose the rule that does the maximum benefit. • Repeat until no rule improves the overall analyses of the training corpus.

  25. Evaluation and Complexity • Training corpus 4892 word tokenslearned 93 rules.errors 14.5%  6.2% • Complexity of the learning algorithm O(c3), where c = size of the training corpus. • Complexity of the correction O(r·n), where r = number of rules, n = size of trial text.

  26. The sentence stage • Use a syntactic parser to rule out improbable analyses. • The pair stage – adjacent words, the sentence stage – long term dependencies.

  27. Example • מורה הכיתה הנמוכה נכנס לכיתהMWRH HKITH HNMWKH NKNS LKITH more/mora ha-kitta ha-nmuka niknas … masc/fem verb-masc more ha-kitta ha-nmuka niknas …

  28. S VP NP COMP V COMP N N J PREP N more ha-kitta ha-nmuka niknas la-kitta Score of a syntax tree score(s) = score(more)score(ha-kitta)  … score(la-kitta) The challenge: calculate the score of all syntax trees without enumerating all trees

  29. Dynamic Programming • Table[i,j,A] = the maximum score of all parses • Fill table by incrasing values of Time complexity

  30. Word Stage 14 7 Pair Stage Sentence Stage 5.3 3.8 21 36 20 14 Evaluation error rate

More Related