sdc the spoken dialog challenge l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SDC: The Spoken Dialog Challenge PowerPoint Presentation
Download Presentation
SDC: The Spoken Dialog Challenge

Loading in 2 Seconds...

play fullscreen
1 / 9

SDC: The Spoken Dialog Challenge - PowerPoint PPT Presentation


  • 163 Views
  • Uploaded on

SDC: The Spoken Dialog Challenge. Maxine Eskenazi Alan Black. SDC: the Spoken Dialog Challenge. Organized by the Dialog Research Center (NSF-sponsored) Guided by Community input Uses a common platform to compare research techniques to compare evaluation techniques

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SDC: The Spoken Dialog Challenge' - toby


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sdc the spoken dialog challenge

SDC: The Spoken Dialog Challenge

Maxine Eskenazi

Alan Black

sdc the spoken dialog challenge2
SDC: the Spoken Dialog Challenge
  • Organized by the Dialog Research Center (NSF-sponsored)
  • Guided by Community input
  • Uses a common platform
    • to compare research techniques
    • to compare evaluation techniques
    • to compare simulated users
  • Share:
    • Data
    • System (optional)
    • Access to real users
sdc 2010
SDC 2010
  • October 2009: Training webcasts
  • January 2010: Data distributed
  • January 2010 - April 2010: Development period
  • May 2010: Control tests
  • July - August 2010: Live tests
  • December 2010: Reporting: Special Session SLT
getting started
Getting started
  • Task: bus information using the Let’s Go system with real users
  • Dialog data distributed:
    • 700GB (1 year of data, ~14,000 dialogs)
      • Labeled (crowdsourcing and some expert)
    • Detailed logs and audio
  • Base system distributed:
    • Olympus, training sessions, but participants could use their own systems
what happened
What happened?
  • Participants:Academia and industry, internationally
  • Four systems in control test
  • Three systems in live test
  • Three evaluation schemes
  • One simulated user system
  • Scheme to transfer calls to other countries (thanks to ATT!)
  • Want to know the results? Come to SLT!
sdc 2011
SDC 2011
  • Same application
    • Allows those who did not participate this year to join
    • “first pancake” this year
    • Will allow comparison over two years
    • Looking for new applications with real users for SDC 2012
  • Aiming at publications and a teaching tool – two tracks
sdc 2011 schedule
SDC 2011 Schedule
  • Track 1 SDC 2011 Track 2SDC 2011

Primary TrackSecondary track

  • 2010 Nov Development2010 Dec   Development2011 Jan   Development2011 Feb   Development2011 Mar   Control Tests2011 Apr   Control TestsDevelopment           Dry Runs 2011 May   CT labeling       Development2011 Jun   CT results        DevelopmentLive Test/Sim2011 Jul   Live Test/SimDevelopment2011 Aug   Live Test/SimControl Tests2011 Sep   LT labelling      Control Tests                             Dry runs 2011 Oct   LT results        CT labeling EvaluationSDC2012Stream 12011 Nov   Evaluation        CT Results          DevelopmentLive Test/SimDevelopment2011 Dec   EvaluationLive Test/SimDevelopment2012 Jan                     Live Test/SimDevelopment2012 Feb                     LT labeling         Control Tests2012 Mar                     LT results          Control TestsEvaluation          ... 2012 Apr                     Evaluation2012 May                     Evaluation
slide8
SDC
  • Questions?
slide9
max@cmu.edu
  • awb@cs.cmu.edu