Tips for writing gooder grammars
Download
1 / 19

Tips for Writing Gooder Grammars - PowerPoint PPT Presentation


  • 137 Views
  • Updated On :

Tips for Writing Gooder Grammars. Judi Halperin Speech Engineer. Overview. Why does the grammar development process matter?. Grammar Development Lifecycle Best practices What not to do Why?. Avoid the pitfalls!!. A successful ASR application requires…. A good design….

Related searches for Tips for Writing Gooder Grammars

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Tips for Writing Gooder Grammars' - damaris


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Tips for writing gooder grammars l.jpg

Tips for Writing Gooder Grammars

Judi Halperin

Speech Engineer


Overview l.jpg
Overview

  • Why does the grammar development process matter?

  • Grammar Development Lifecycle

    • Best practices

    • What not to do

    • Why?

  • Avoid the pitfalls!!


A successful asr application requires l.jpg
A successful ASR application requires…

A good design…

Application Development

…efficient coding…

Design

Infrastructure

Grammars

…and effectively constrained grammars!!!

…supporting infrastructure…


Bottom line l.jpg
BOTTOM LINE…

If the system does not recognize what callers say, they will not be able to complete their task in the IVR!!

  • As a result…

    • Caller/Customer satisfaction will be

    • negatively impacted

  • Containment rate heads south and

  • ROI goes with it!



Design and plan l.jpg
Design and Plan

  • Should be started in parallel with VUI design

  • Determine feasibility

    • Can the grammar be developed?

    • Can the production hardware support it in real time?

  • Start large efforts early

  • Set expectations early

    • If caller identification requires alphanumeric recognition, make sure your customer understands the recognition capabilities early on!

  • Host modification takes time

  • Dynamic and other complex grammars require more testing

  • New concepts and/or languages may require external resources

    • A prescription names grammar will typically require both a large grammar AND corresponding lexicon development.

  • Allow time for collaboration with VUI designer and developer


Creating your menu grammar l.jpg
Creating Your Menu Grammar

  • Don’t include everything a caller might say, but rather what a cooperative caller would say

    • Leave the addition of possible synonyms to tuning

    • The more in-grammar items, the more chance of false accepts

  • Consider logical synonyms based on the prompt

    • Callers have a tendency to parrot back the prompt:

      • PROMPT: “..or say ‘main menu’ to start over.”

      • Callers would be likely to say “main menu”, “start over” or “main menu to start over”

  • Include the actual spelling of the word or phrase in the grammar to take advantage of the pronunciations in the system dictionary

    • “that’s” not “thats”

    • “t v” not “teevee”

    • “don’t” not “dont”

    • “f a q” not “f a queue” (it’s just silly)


General grammar development tips l.jpg
General Grammar Development Tips

  • Minimize the use of pre-and-post-fillers out of the box

    • PROMPT: “What’s your account number?”

    • Callers are likely to say “it’s account number...” but not “I think it might be...”

    • Leave the addition of these items to tuning once sufficient data proves their relevance

  • Choose your built-ins wisely

    • Control your content when appropriate

    • Don’t use alphanumeric built-in grammar for spelling-only input

      • Unnecessary overhead can have a latency effect

      • Overgeneration leads to decrease in recognition accuracy (and possible latency)

    • Beware: Some built-ins overgenerate


Task allow spelling using military alphabet l.jpg
Task: Allow spelling using military alphabet

<rule id="root">

<item><ruleref uri="letter"><tag>ASCII=letter.LETTER</tag></item>

<item repeat="0-1">as in</item>

<item>

<ruleref uri="militaryAlpha"><tag>MIL = militaryAlpha.MIL</tag>

</item>

</rule>

<rule id="letter">

<one-of>

<item>a

<tag>LETTER="a"</tag>

</item>

<item>b

<tag>LETTER="b"</tag>

</item>

… ..

<item>y

<tag>LETTER="y"</tag>

</item>

<item>z

<tag>LETTER="z"</tag>

</item>

</one-of>

</rule>

<rule id="militaryAlpha">

<one-of>

<item>alpha

<tag>MIL="a"</tag>

</item>

<item>bravo

<tag>MIL="b"</tag>

</item>

… .

<item>yankee

<tag>MIL="y"</tag>

</item>

<item>zulu

<tag>MIL="z"</tag>

</item>

</one-of>

</rule>


Task allow spelling using military alphabet10 l.jpg
Task: Allow spelling using military alphabet

<rule id="root">

<item><ruleref uri="letter"><tag>ASCII=letter.LETTER</tag></item>

<item repeat="0-1">as in</item>

<item>

<ruleref uri="militaryAlpha"><tag>MIL = militaryAlpha.MIL</tag>

</item>

</rule>

This grammar generates “a as in alpha”…

<rule id="letter">

<one-of>

<item>a

<tag>LETTER="a"</tag>

</item>

<item>b

<tag>LETTER="b"</tag>

</item>

… ..

<item>y

<tag>LETTER="y"</tag>

</item>

<item>z

<tag>LETTER="z"</tag>

</item>

</one-of>

</rule>

<rule id="militaryAlpha">

<one-of>

<item>alpha

<tag>MIL="a"</tag>

</item>

<item>bravo

<tag>MIL="b"</tag>

</item>

… .

<item>yankee

<tag>MIL="y"</tag>

</item>

<item>zulu

<tag>MIL="z"</tag>

</item>

</one-of>

</rule>

…but also “a as in zulu”!!


Consider all methods of constraint l.jpg
Consider all methods of constraint

  • Avoid overgeneration when possible

    • Don’t “sort it out” on the application side if you don’t have to

      • But watch out for introduced latency

    • Don’t allow account numbers that are not valid

    • Make sure your pre-and-post-fillers make sense at every invocation:

      • “I want my account balance” but not “I want my help”

    • Make sure your in-grammar items make sense at every invocation:

      • “a as in alpha” but not “a as in zulu”

      • Constrain birth dates to dates before today’s date and make assumptions based on caller demographics as to the earliest allowable date


Remove the overgeneration l.jpg
Remove the overgeneration…

<rule id="letter">

<one-of>

<item>a as in alpha

<tag>LETTER="a"</tag>

</item>

<item>b as in bravo

<tag>LETTER="b"</tag>

</item>

.

.

<item>y as in yankee

<tag>LETTER="y"</tag>

</item>

<item>z as in zulu

<tag>LETTER="z"</tag>

</item>

</one-of>

</rule>

<rule id="root">

<item>

<ruleref uri="letter">

<tag>ASCII=letter.LETTER</tag>

</item>

</rule>

This grammar generates “a as in alpha”…

…but NOT “a as in zulu”!!


You have the power l.jpg
You have the power!

  • Some platforms allow scripting within the grammar itself

  • Grammar-internal scripting is useful in many situations

  • Further constrain dates, times, currency, etc

  • Implement a skip list or deweight a previous response that was negatively confirmed

  • Apply a checksum to disallow invalid digit string input

  • Calculate additional information based on the caller’s response

    • Day of the week given the date

    • Determine the time given colloquial response, ex: “in an hour”


Constrain using javascripting l.jpg

<rule id="root">

<item><ruleref uri="letter">

<tag>tmpL=letter.LETTER</tag>

</item>

<item repeat="0-1">as in</item>

<item><ruleref uri="militaryAlpha">

<tag>tmpM=militaryAlpha.MIL</tag>

</item>

<tag>

<![CDATA[

if (tmpL != tmpM) {

SWI_DISALLOW = 1;

} else {

LETTER = tmpL;

}

]]>

<tag>

</rule>

<rule id="letter">

<one-of>

<item>a <tag>LETTER="a"</tag></item>

<item>b <tag>LETTER=“b"</tag></item>

.

.

<item>y <tag>LETTER=“y"</tag></item>

<item>z <tag>LETTER="z"</tag></item>

</one-of>

</rule>

<rule id="militaryAlpha">

<one-of>

<item>alpha <tag>MIL="a"</tag></item>

<item>bravo <tag>MIL="b"</tag></item>

.

<item>yankee <tag>MIL="y"</tag></item>

<item>zulu <tag>MIL="z"</tag></item>

</one-of>

</rule>

Constrain using javascripting…


Simplify continuous string recognition l.jpg
“Simplify” Continuous String Recognition

  • Constrain digits, alphanumeric and spelling grammars as tightly as possible

  • Load valid strings dynamically (especially for alphanumeric and spelling grammars)

  • If using a static grammar, apply appropriate patterns

  • Positively weight patterns that occur more often (if appropriate)

  • Apply logic to the task

  • If collecting the 1st two letters of a last name:

    • Use census data or other appropriate information to help constrain to the top 90+% of last names in the US

  • Don’t allow letter combinations that are not reasonably expected to occur given the task, or even producible by a native speaker, ex: “QB” or “CC”


Test before throwing it over the fence l.jpg
Test Before Throwing it Over the Fence

  • Open the grammar in a web browser or XML editor to validate

  • Use platform tools to test the grammar

    • Try both valid and invalid entries

    • Verify correctness of the slot values

      • Validate the mappings from raw text to slot value

      • Validate any calculated values returned in slots

    • Generate a large corpus, extract unique entries and validate

  • Load the grammar into a test application and give it a spin

    • Don’t worry about unit testing OOGs – leave that to tuning


Pilot test and tune l.jpg
Pilot Test and Tune

  • Collect a sufficiently large data set over various times, dates and demographics

    • Not just high traffic or low traffic periods

    • Not during a one-time promotion

    • Not only when callers might have a higher tendency towards otherwise infrequently used functionality

      • Tax season, beginning of the month, etc.

  • Transcribe what the caller said

  • Analyze performance and make recommendations for improvements

  • Tuning should be performed periodically as appropriate


Avoid grammar development pitfalls l.jpg
Avoid Grammar Development Pitfalls!!

Include actual spellings…

Use grammar-internal scripting if appropriate…

Validate your grammar….

‘Simplify’ continuous string recognition…

  • Start large efforts early…

Avoid overgeneration…

Minimize out of-the-box pre-and-post-fillers

Generate custom pronunciations if needed…

Plan it out…

Constrain as tightly as possible…

Include only what a cooperative caller would say…

Test, test, test….

Pay attention to the prompt…

Use built-ins as appropriate…

Pilot test and tune….


Slide19 l.jpg

Judi Halperin

[email protected]

19

© 2007 Avaya Inc. All rights reserved.

Avaya – Proprietary & Confidential. Under NDA


ad