1 / 24

Presented at the ANLP-NAACL 2000 Embedded Machine Translation Systems Workshop The MEI Team

Mandarin-English Information (MEI): Investigating Translingual Speech Retrieval Johns Hopkins University Summer Workshop 2000. Presented at the ANLP-NAACL 2000 Embedded Machine Translation Systems Workshop The MEI Team. MEI Team. Helen Meng Chinese University of Hong Kong

briana
Download Presentation

Presented at the ANLP-NAACL 2000 Embedded Machine Translation Systems Workshop The MEI Team

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mandarin-English Information (MEI):Investigating Translingual Speech RetrievalJohns Hopkins University Summer Workshop 2000 Presented at the ANLP-NAACL 2000 Embedded Machine Translation Systems Workshop The MEI Team

  2. MEI Team Helen Meng Chinese University of Hong Kong Erika Grams Advanced Analytic Tools Sanjeev Khudanpur Johns Hopkins University Gina-Anne Levow University of Maryland Douglas Oard University of Maryland Patrick Schone US Department of Defense Hsin-Min Wang Academia Sinica, Taiwan • Senior Members • Students Berlin Chen National Taiwan University Wai-Kit Lo Chinese University of Hong Kong Karen Tang Princeton University Jianqiang Wang University of Maryland

  3. Outline • Audio indexing • MEI Project overview • Research challenges • System architecture • Collaboration opportunities

  4. Motivation • Speech retrieval applications are emerging • e.g., http://speechbot.research.compaq.com • Internet-accessible • Radio and • Television • Stations source: www.real.com, Feb 2000

  5. The Big Picture MEI Translingual Audio Search Translingual Audio Browsing Speech to Speech Translation Select Examine English Query English Audio

  6. Related Work • TREC Spoken Document Retrieval • Close coupling of recognition and retrieval • TREC Cross-Language Retrieval • Close coupling of translation and retrieval • TDT-3 Topic Tracking • Coupling recognition, translation and retrieval • Using speech recognition transcripts

  7. Mandarin Audio Collection Query by Example English Example Newswire Stories The MEI Project • Closely couple recognition and translation • For the purpose of retrieval • Using English examples, find Mandarin audio

  8. Research Challenges • Multi-scale audio indexing • Multiple feature sets capture more information • Multi-scale translation • Lexicon and pronunciation are complementary • Multi-scale retrieval • Combination of evidence can add robustness

  9. /j/ /ng/ Preme/Toneme /i/ /a/ /ji/ /ang/ Preme/Core Final /j/ /iang/ Initial/Final Multi-scale Mandarin Audio Indexing

  10. Multi-scale Translation • Word-scale • Dictionary-based [Levow & Oard 00] • Parallel corpora [Nie 99] • Comparable corpora [Fung 98] • Subword-scale [Knight & Graehl 97] • Cross-language phonetic mapping • /bei2 ai4 er3 lan2/ • Kosovo (/ke1-sou3-wo4/, /ke1-sou3-fo2/, /ke1-sou3-fu1/, /ke1-sou3-fu2/)

  11. Cross-Language Phonetic Mapping • Syllabify English spelling • e.g. Jiang Zemin, Shandong Province • Map English pronunciation to Mandarin • Convert phonemes to pinyin • e.g. /k ow s ax v ow/ to /ke1-suo3-wo4/ • Plan to investigate alternative techniques • Rule-based • Statistical mapping

  12. Multi-scale Retrieval • Word-scale exploits lexical knowledge • Enhances precision • Subwords can achieve complete coverage • Enhances recall • Combination of evidence may be best • If a good merging strategy can be found

  13. Multi-scale Retrieval Techniques • Subword-scale • Syllable lattice matching [Chen, Wang & Lee 00] • Overlapping syllable n-grams [Meng et al. 99] • Syllable confusion matrix [Meng et al. 99] • Word-scale • Structured queries [Pirkola 98] • Structured translation [Sperer & Oard 00]

  14. Merging Strategies • Loose coupling • Separate retrieval runs • Merge ranked lists [Voorhees 95] • Tight coupling [Ng 00] • Unified indexing of words and subwords • Single ranked list

  15. Robust Retrieval • Multiple causes • Speech recognition errors • Translation ambiguity • Transliteration ambiguity • Possible solutions • Weighted n-best indexing [Levow & Oard 00] • Syllable lattice indexing [Chen, Wang & Lee 00] • Syllable confusion expansion [Meng et al. 99] • Structured queries [Pirkola 98] • Document expansion [Levow & Oard 00]

  16. System Architecture Overview Mandarin Documents Syllable n-grams Words Corpus Statistics Translation Lexicon Relevance Judgments Known Terms Word Translation Retrieval System Eval Code Average Precision English Example Phonetic Transcription Syllable n-gram Generation

  17. The TDT Collections • Four stories per topic in each language • Each reporting on some aspect of one event Development Test (TDT-2) Evaluation (TDT-3) Mar 98 Jun 98 Oct 98 Dec 98 41 Hours VOA Mandarin Audio 121 Hours Voice of America (VOA) Mandarin Audio Story Boundaries Known Condition Jan 98 Associated Press (APW) New York Times (NYT) English Newswire APW+NYT English 20 Topics 59 Topics

  18. MEI Project Schedule Second Team Planning Meeting First Team Planning Meeting Workshop 00 Planning Meeting Dec Feb Apr Jun Aug Six Weeks at Hopkins:

  19. Things We Need • Ideas • To sharpen our focus • Connections • To build a community of interest • Resources • To build on what others have done

  20. For More Information • MEI Project • http://www.glue.umd.edu/~meiweb • Translingual Retrieval • http://www.clis.umd.edu/dlrg/clir • Speech Retrieval • http://www.clis.umd.edu/dlrg/speech • Hopkins Summer Workshop Series • http://www.clsp.jhu.edu/workshops

  21. Detailed Query Processing (1) Terms that have Mandarin translations List of translatable words and phrases White-space separated text with named entity tags. Stopping Stemming Phrase Extraction Named entities terms with no Mandarin translations

  22. Detailed Query Processing (2) Terms that have Mandarin translations xxxxx xx xxx xxx zzzzzzz ww xx ww z rrrrrr sss ttttt tt ww www term translation Named entity parsing rules for transliteration English-Mandarin translation lexicon Northern Ireland Named entities term translation Named entity parsing Bag of Mandarin terms phonetic expansion English pronunciation lexicon eh n t ih t iy t eh k s t Bag of English phone sequences Terms with no Mandarin translations phonetic expansion

  23. Detailed Query Processing (3) xxxxx xxx zzzzzzz xx ww z rrrrrr sss ttttt tt ww www no To trash (or downweight) Bag of Mandarin terms Retain this term? yes ASR substitution/deletion prone words ASR Insertion prone words Smaller bag of Mandarin terms Syllabic expansion s1 s2 s2 s5 s1 s2 s3 Mandarin syllabification rules Mandarin pronunciation lexicon Bag of English phone sequences Two bags of Mandarin syllable sequences English phone strings to Mandarin syllables sa sb sc sd sc se

  24. Detailed Query Processing (4) Bag of high-confidence Mandarin terms xxxxx xxx zzzzzzz xx ww z rrrrrr sss ttttt tt ww www s0 s1 s1 s2 s2 s3 s3 s4 s3 s5 s5 s6 s5 s7 s7 s7 Syllable n-gram generation syllabic expansion Mandarin pronunciation lexicon Bag of Mandarin lexical terms s1 s2 s2 s3 s2 s5 s5 s1 Syllable n-gram generation Mandarin syllable sequences from likely recognition errors Three Bags of Mandarin syllable n-grams from different sources sa sb sb sc sc sd sc se Syllable n-gram generation Mandarin syllable sequences from unknown words

More Related