SUNY Albany Tasks

1. SUNY Albany Tasks • US Coordinating Partner (WP #0) • Main contributions to WP #2, 4, 5 • Dialogue Annotation (English) – WP #2 • Dialogue Annotation tool • Transaction semantics • Robust NLP – in WP #4 • Utterance to query conversion • Dialogue Manager – WP #5 • Frame Router • Mixed-initiative dialogue • Support for WP #6 • Interface to Customer Database – in WP #6 • Partial-match access & verification AMITIES Meeting Sheffield

2. Accomplishments as of 05/02 • Dialogue Annotation (WP #2) • Completed revised edition of the DialogueTool • Supports functional (DAMSL) & semantic annotation • DT distributed to Consortium for use • Created initial set of transaction frames • Based on Leeds dialogues AMITIES Meeting Sheffield

3. Accomplishments as of 05/02 • Dialogue Manager (WP #5) • Implemented for transaction and caller id • Initially limited to a handful of caller transactions • Plus a number of supporting transactions • Supports mixed initiative dialogue • Identify transaction first • Collect necessary attributes values • Can happen in any order the caller chooses • Implemented as Galaxy server • Initial development on Solaris • Transferred to Linux  target hardware AMITIES Meeting Sheffield

4. Accomplishments as of 05/02 • Utterance  query conversion (WP #4) • Extract key elements of caller utterance • Transaction extraction • Address change, Lost card, etc. – by profile • Attribute extraction • Customer name, account, address, etc. – IE • Implemented through NLP server • Uses Sheffield’s GATE system • IE routines developed at Sheffield AMITIES Meeting Sheffield

5. Accomplishments as of 05/02 • Interface to Cust. Database (WP #6) • Requires NLP server to identify attributes • GATE delivers attribute annotation • Queries formed to PostgreSQL database • Partial match achieved by wildcard matching of column contents • This supports recovery from ASR errors AMITIES Meeting Sheffield

6. Caller Transactions • A caller’s purpose is to perform a certain transaction or transactions: • Address change (update info) • Balance inquiry (find info) • Report lost card (new info) • … many others • Transactions typically carry attributes: • Key: name, account#, address, post code, … • Supporting: credit limit; branch name; … • Simultaneous access to database needed: • To verify what caller says – partial match required • To recover from misunderstandings AMITIES Meeting Sheffield

7. Agent Transactions • An agent is also performing transactions: • As part of routine call center procedure • E.g., verify caller’s identity • To communicate certain information to the caller • Confirm changes, convey decisions (accept/reject), etc. • Agent transactions may be interspersed with caller transactions • E.g. Verify-Id may be necessary to proceed with most caller transactions • Side-effect: finding attribute values AMITIES Meeting Sheffield

8. Sample transactions & attributes AMITIES Meeting Sheffield

9. Example: address change AGENT:[Conv.Opening/STrans=GreetCaller; CTrans=undef] Good afternoon, Customer Services, how can I help? CUST: [Action-Directive/CTrans=UpdateAddr] Hello I wanted to notify you of a change of address AGENT: [Info-Request/STrans=VerifyId] Yes certainly, can I take your name? CUST:[Answer/STrans=VerifyId; CName=’Jodie Smith’] Yeh, its Miss Jodie Smith AMITIES Meeting Sheffield

10. How does it work? Process utterance Identify user transaction Select agent transaction Generate utterance Respond to the caller Select attributes to ask for Conversation manager AMITIES Meeting Sheffield

11. DM Implementation • We use a collection of synchronous processes • These processes keep track of system goals: • corresponding to caller-initiated transactions • corresponding to system-initiated transactions • Example processes: • Greet Caller (system-init default) • Identify Caller Transaction (system-init default) • Identify Caller (if required by transaction) • other …. • A process runs until: • It’s goal is achieved • It is rendered irrelevant by other events AMITIES Meeting Sheffield

12. Amitiés System Architecture Text-to-speech Conversion Speech Recognition Audio Server Nat’l Language Understanding Hub French Database Server German Dialogue Manager Response Generation Engl. CustomerDatabase French German LIMSI Sheffield SUNY AMITIES Meeting Sheffield

13. Dialogue Tool • An updated version available • Includes recommendations from Paris meeting (at LIMSI) • Detailed review tomorrow • Wholesale annotation effort starts: • At SUNY & Duke – English • At LIMSI – French • German: sample annotations by Gert W. AMITIES Meeting Sheffield

14. AMITIES Meeting Sheffield

15. Example: Destroyed Card AGENT [ConvOpening/CTrans=undef; STrans=GreetCaller] Good afternoon, Customer Services, how can I help? CUST: [Assert/CTrans=undef; Name=‘Smith’;AccNo=‘123’] Hello Mrs Smith here, the account number is 123 AGENT:[Info-Request/CTrans=undef; STrans=PromptCTrans] Thanks and how can I help? CUST:[Action-Directive/CTrans={LostCard; DestroydCard, …}] It’s to order a replacement card AGENT:[Info-Request/STrans=Clarify(CTrans)] OK has it been lost or… CUST:[Answer/CTrans=DestroyedCard] destroyed AMITIES Meeting Sheffield

16. Albany: next 6 months • Annotate call center dialogues (WP #2) • The goal is 10K dialogues • Dialogue Manager (WP #5) • Complete first year demo system • Expand to additional transactions • More caller transactions (but finite set) • More agent transactions • Clarification subdialogues • Move beyond transaction & caller id • Execute transactions (eg. complete UpdateAddr) • Response Generator • Replace canned responses now • Utterance to query conversion (WP #4) • Incorporate French and German AMITIES Meeting Sheffield