The mei team july 17 2000
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

The MEI Team July 17, 2000 PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on
  • Presentation posted in: General

Mandarin-English Information (MEI): Investigating Translingual Speech Retrieval Johns Hopkins University Center of Language and Speech Processing Summer Workshop 2000 Opening Ceremony. The MEI Team July 17, 2000. MEI Team. Senior Members Students.

Download Presentation

The MEI Team July 17, 2000

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The mei team july 17 2000

Mandarin-English Information (MEI):Investigating Translingual Speech RetrievalJohns Hopkins University Center of Language and Speech ProcessingSummer Workshop 2000Opening Ceremony

The MEI Team

July 17, 2000


Mei team

MEI Team

  • Senior Members

  • Students

Helen Meng Chinese University of Hong Kong

Erika Grams Advanced Analytic Tools

Sanjeev Khudanpur Johns Hopkins University

Gina-Anne Levow University of Maryland

Douglas Oard University of Maryland

Patrick Schone US Department of Defense

Hsin-Min Wang Academia Sinica, Taiwan

Berlin Chen National Taiwan University

Wai-Kit Lo Chinese University of Hong Kong

Karen Tang Princeton University

Jianqiang Wang University of Maryland


Outline

Outline

  • Motivation

  • MEI project overview

  • Research challenges

  • System architecture


Motivation

Motivation

  • Monolingual speech retrieval applications are emerging, e.g.

    • http://speechbot.research.compaq.com

Internet-accessible

Radio and Television

Stations

source: www.real.com, Feb 2000


Translingual speech retrieval

Translingual Speech Retrieval

  • Allow anyone to find information that is expressed in any spoken language


The big picture

English Text Query

Mandarin News Broadcasts

English-to-Chinese

Translation

Mandarin Audio

Indexing

Retrieval Engine

Retrieved Mandarin

Spoken Documents

Search

Refinements

Speech-to-Speech

Translation

English Spoken Documents

The Big Picture


Related work

Related Work

  • TREC Spoken Document Retrieval

    • Close coupling of recognition and retrieval

  • TREC Cross-Language Retrieval

    • Close coupling of translation and retrieval

  • TDT-3 Topic Tracking

    • Coupling recognition, translation and retrieval

    • Using speech recognition transcripts


Research challenges

Research Challenges

The MEI Project

  • Closely couple recognition and translation

    • for the purpose of retrieval

  • Multi-scale audio indexing

    • Multiple feature sets capture more information

  • Multi-scale translation

    • Lexicon and pronunciation are complementary

  • Multi-scale retrieval

    • Combination of evidence can add robustness


Task and corpus

Mandarin

Audio

Collection

Query by

Example

English

Example

Newswire

Stories

Task and Corpus

  • TDT Corpus

  • Challenge: using an English story as the query, find related Mandarin audio documents


Multi scale mandarin processing phonological considerations

Multi-scale Mandarin Processing: Phonological Considerations

  • Chinese is a syllable-based language

  • Mandarin is the official Chinese dialect

    • ~400 base syllables, 4 lexical tones + light tone

  • Syllable structure (CG)V(X)

    • (CG): onset, optional, consonant+medial glide

    • V:nuclear vowel

    • X:coda, glide / alveolar nasal / velar nasal

    • ~ 21 initials, 39 finals

  • Circumvent the OOV problem


Multi scale mandarin processing linguistic considerations

Multi-scale Mandarin Processing:Linguistic Considerations

  • Characters (written) -> syllables (spoken)

  • Degenerate mapping

    • /hang2/, /hang4/, /heng2/ or /xing2/

    • /fu4 shu4/ (LDC’s CALLHOME lexicon)

  • Tokenization / Segmentation

    • /zhe4 yi1 wan3 hui4 ru2 chang2 ju3 xing2/


Multi scale audio indexing

/j/

/ng/

Phones

/i/

/a/

/ji/

/ang/

Preme/Core Final

/j/

/iang/

Initial/Final

Multi-scale Audio Indexing

Preme/Toneme, Initial/Tonal-Final

syllable /jiang/, single-character word, part of a multi-char word


Multi scale translation

Multi-scale Translation

  • Word-scale

    • Phrase-based, e.g. “the White House”

    • Dictionary-based [Levow & Oard 00]

    • Lexical gaps: parallel corpora, comparable corpora

  • Subword-scale

    • Named entities: transliteration by cross-lingual phonetic mapping

    • Northern Ireland /bei2 ai4 er3 lan2/

    • Kosovo (/ke1-suo3-wo4/, /ke1-suo3-fo2/, /ke1-suo3-fu1/, /ke1-suo3-fu2/)


Cross lingual phonetic mapping

Cross-Lingual Phonetic Mapping

Named entity Jiang Zemin, Kosovo

Syllabify Pinyin Spelling

E.g. jiang ze min

English Pronunciation Lookup

or

Letter-to-Phone Generation

English Phones, e.g. k ao s ax v ow

Cross-lingual Phonetic Mapping

Chinese Phones, e.g. k e s u o w o

Syllabification

Chinese syllables, e.g. ke suo wo


Multi scale retrieval

Multi-scale Retrieval

  • Word-scale exploits lexical knowledge

    • Enhances precision

  • Subwords can achieve complete coverage

    • Enhances recall

  • Combination of evidence may be best

    • If a good merging strategy can be found


Merging strategies

Merging Strategies

  • Loose coupling

    • Separate retrieval runs

    • Merge ranked lists [Voorhees 1995]

  • Tight coupling

    • Unified indexing of words and subwords

    • Single ranked list

    • [Ng 2000]


Multi scale retrieval techniques

Multi-scale Retrieval Techniques

  • Subword-scale

    • Syllable lattice matching [Chen, Wang & Lee 2000]

    • Overlapping syllable n-grams [Meng et al. 1999]

    • Syllable confusion matrix [Meng et al. 1999]

  • Word-scale

    • Structured queries [Pirkola 1998]

    • Structured translation [Sperer & Oard 2000]

  • Robust Retrieval

    • speech recognition errors

    • translation / transliteration ambiguities


Research plan

Research Plan

  • “MEI: Mandarin-English Information”, Proceedings of the TDT Workshop, 2000.

  • “Mandarin-English Information (MEI):Investigating Translingual Retrieval” Proceedings of the NAACL Workshop on Embedded Machine Translation, 2000


The mei team july 17 2000

Mandarin-English Information: Investigation Translingual Speech Retrieval<http://www.glue.umd.edu/~meiweb>

Johns Hopkins University, Center for Language and Speech Processing, JHU/NSF Summer Workshop 2000

MEI Team :Helen MENG (CUHK), Berlin CHEN (National Taiwan University), Erika GRAMS (Advanced Analytic Tools), Sanjeev KHUDANPUR (JHU/CLSP), Gina-Anne LEVOW (University of Maryland), Wai-Kit LO (CUHK)

Douglas OARD (University of Maryland), Patrick SCHONE (Department of Defense), Karen TANG (Princeton University), Hsin-Min WANG (Academia Sinica), Jianqiang WANG (University of Maryland)

Input English text query

English to Chinese translation dictionary

Word sequence

Translated words and phrases

Baseline

Character

n-gram sequence

Phrase tagging

Term Translation

Character n-gram generation

As of Sunday July 9, 2000

Query Processing

Unknown words and phrases

Query to INQUERY

Named Entity Tagger

Query Term Selection

INQUERY

Ranked List of Possibly Relevant Documents

Figure of Merit

Document to INQUERY

Scoring

Character

n-gram sequence

Document Processing

Character n-gram generation

Relevance Assessments

Dragon Mandarin ASR

Spoken Mandarin documents

Word sequence

Segmented Chinese Text


The mei team july 17 2000

END


  • Login