Experiments on building language resources for multi modal dialogue systems
Download
1 / 1

Experiments on Building Language Resources for Multi-Modal Dialogue Systems - PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on

User Scenarios useful in the context of lacking real human-machine interactions; designed to obtain homogeneous linguistic coverage, for all the languages: several styles (or registers - familiar, elaborated); specific phrases (politeness phrases, time intervals - "from the sixties");

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Experiments on Building Language Resources for Multi-Modal Dialogue Systems' - borna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Experiments on building language resources for multi modal dialogue systems

  • User Scenarios

  • useful in the context of lacking real human-machine interactions;

  • designed to obtain homogeneous linguistic coverage, for all the languages:

    • several styles (or registers - familiar, elaborated);

    • specific phrases (politeness phrases, time intervals - "from the sixties");

    • various syntactic components (passive constructions, relative clauses, questions and ellipses);

    • dates or names

  • developers worked independently

  • building exhaustive user scenarios

  • Context-free grammar

  • covers the linguistic phenomena from the user scenario, for every language

  • Technical vocabulary

  • covers the linguistic phenomena from the user scenario, for every language

  • Adapting parser’s resources

  • TAG widely accepted formalism for syntactic parsing

  • XML standard for grammars (TAGML) and for semantic representations (MMIL)

  • existing resources for English, French for free texts

  • Speech recognizer’s language model

  • Statistical language model

  • 400 words vocabulary

  • Huge number of possible sentences

  • Training corpus: generated with context-free grammar. Two steps:

    • Bigram model using classes (ex: P(DECADES|the)

    • Uniform distribution of words into classes: P(90’s|DECADES)

Developping the language resources

L.Romary*, A.Todirascu**, D.Langlois*

*LORIA, Nancy, France

**Université de Technologie de Troyes, France

Experiments on Building Language Resources for Multi-Modal Dialogue Systems

  • Goals

  • identification of a methodology for adapting linguistic resources for human-machine dialogue systems, without training corpora;

  • Multilinguality supported, uniform linguistic coverage of speech interpretation modules

  • case study: synchronising parser’s and SR interface for French for MIAMM project

  • The Multimedia Information Access Using Multiple Modalities (MIAMM) Project (2002-2003)

  • A prototype of a human-machine dialogue interface regrouping various interaction modalities: speech, haptics, graphics;

  • Case study: searching music into an existing database using all the modalities

  • Multilinguality supported: English, French , German

  • difficulties: multi-modal training corpora not available, Information flow: several models for the same language (one for speech, one for parsing : how to cover the same language),changing specifications during the project

  • Parser’s Experiments

  • iteration updating process after interaction with other modules

  • adding new lexical entries

  • local grammars

    • generated by a meta-grammar

    • preference for substitution

    • Domain-specific syntactic components (elliptic phrases, navigation verbs, noun groups)

  • mapping lexical entries to domain ontology

  • transforming derivation trees into MMIL representations

  • Speech Recognition’s Experiments

  • system: ESPERE (small vocabularies)

  • protocol: 88 sentences recorded, 4 speakers, 2 men and 2 females, no OOV, all sentences in language

  • several methods for estimating P(w|v)

    • Frequencies in training corpus: WER = 3.8, SD = 0.9

    • Uniform probabilities for bigrams in training corpus: WER = 4.0, SD = 0.4

      • True frequencies are not useful

    • All bigrams are possible (not null probabilities, back-off with ): WER >> 4.0

      • Constraint bigrams from grammar are necessary

    • Training corpus (A) is adaptation corpus for a general newspaper one (B):

      • Linear combination  performance are less good than when A is used alone


ad