experiments on building language resources for multi modal dialogue systems
Download
Skip this Video
Download Presentation
Experiments on Building Language Resources for Multi-Modal Dialogue Systems

Loading in 2 Seconds...

play fullscreen
1 / 1

Experiments on Building Language Resources for Multi-Modal Dialogue Systems - PowerPoint PPT Presentation


  • 52 Views
  • Uploaded on

User Scenarios useful in the context of lacking real human-machine interactions; designed to obtain homogeneous linguistic coverage, for all the languages: several styles (or registers - familiar, elaborated); specific phrases (politeness phrases, time intervals - "from the sixties");

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Experiments on Building Language Resources for Multi-Modal Dialogue Systems' - borna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
experiments on building language resources for multi modal dialogue systems

User Scenarios

  • useful in the context of lacking real human-machine interactions;
  • designed to obtain homogeneous linguistic coverage, for all the languages:
    • several styles (or registers - familiar, elaborated);
    • specific phrases (politeness phrases, time intervals - "from the sixties");
    • various syntactic components (passive constructions, relative clauses, questions and ellipses);
    • dates or names
  • developers worked independently
  • building exhaustive user scenarios
  • Context-free grammar
  • covers the linguistic phenomena from the user scenario, for every language
  • Technical vocabulary
  • covers the linguistic phenomena from the user scenario, for every language
  • Adapting parser’s resources
  • TAG widely accepted formalism for syntactic parsing
  • XML standard for grammars (TAGML) and for semantic representations (MMIL)
  • existing resources for English, French for free texts
  • Speech recognizer’s language model
  • Statistical language model
  • 400 words vocabulary
  • Huge number of possible sentences
  • Training corpus: generated with context-free grammar. Two steps:
    • Bigram model using classes (ex: P(DECADES|the)
    • Uniform distribution of words into classes: P(90’s|DECADES)

Developping the language resources

L.Romary*, A.Todirascu**, D.Langlois*

*LORIA, Nancy, France

**Université de Technologie de Troyes, France

Experiments on Building Language Resources for Multi-Modal Dialogue Systems
  • Goals
  • identification of a methodology for adapting linguistic resources for human-machine dialogue systems, without training corpora;
  • Multilinguality supported, uniform linguistic coverage of speech interpretation modules
  • case study: synchronising parser’s and SR interface for French for MIAMM project
  • The Multimedia Information Access Using Multiple Modalities (MIAMM) Project (2002-2003)
  • A prototype of a human-machine dialogue interface regrouping various interaction modalities: speech, haptics, graphics;
  • Case study: searching music into an existing database using all the modalities
  • Multilinguality supported: English, French , German
  • difficulties: multi-modal training corpora not available, Information flow: several models for the same language (one for speech, one for parsing : how to cover the same language),changing specifications during the project
  • Parser’s Experiments
  • iteration updating process after interaction with other modules
  • adding new lexical entries
  • local grammars
    • generated by a meta-grammar
    • preference for substitution
    • Domain-specific syntactic components (elliptic phrases, navigation verbs, noun groups)
  • mapping lexical entries to domain ontology
  • transforming derivation trees into MMIL representations
  • Speech Recognition’s Experiments
  • system: ESPERE (small vocabularies)
  • protocol: 88 sentences recorded, 4 speakers, 2 men and 2 females, no OOV, all sentences in language
  • several methods for estimating P(w|v)
    • Frequencies in training corpus: WER = 3.8, SD = 0.9
    • Uniform probabilities for bigrams in training corpus: WER = 4.0, SD = 0.4
      • True frequencies are not useful
    • All bigrams are possible (not null probabilities, back-off with ): WER >> 4.0
      • Constraint bigrams from grammar are necessary
    • Training corpus (A) is adaptation corpus for a general newspaper one (B):
      • Linear combination  performance are less good than when A is used alone
ad