speech recognition l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Speech Recognition PowerPoint Presentation
Download Presentation
Speech Recognition

Loading in 2 Seconds...

play fullscreen
1 / 69

Speech Recognition - PowerPoint PPT Presentation


  • 572 Views
  • Uploaded on

Speech Recognition. Components of a Recognition System. Frontend. Feature extractor. Frontend. Feature extractor Mel-Frequency Cepstral Coefficients (MFCCs). Feature vectors. Hidden Markov Models ( HMMs ). Acoustic Observations. Hidden Markov Models ( HMMs ). Acoustic Observations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Speech Recognition' - Jims


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
frontend
Frontend
  • Feature extractor
frontend4
Frontend
  • Feature extractor
  • Mel-Frequency Cepstral Coefficients (MFCCs)

Feature vectors

hidden markov models hmms
Hidden Markov Models (HMMs)
  • Acoustic Observations
hidden markov models hmms6
Hidden Markov Models (HMMs)
  • Acoustic Observations
  • Hidden States
hidden markov models hmms7
Hidden Markov Models (HMMs)
  • Acoustic Observations
  • Hidden States
  • Acoustic Observation likelihoods
acoustic model
Acoustic Model
  • Constructs the HMMs of phones
  • Produces observation likelihoods
acoustic model11
Acoustic Model
  • Constructs the HMMs for units of speech
  • Produces observation likelihoods
  • Sampling rate is critical!
  • WSJ vs. WSJ_8k
acoustic model12
Acoustic Model
  • Constructs the HMMs for units of speech
  • Produces observation likelihoods
  • Sampling rate is critical!
  • WSJ vs. WSJ_8k
  • TIDIGITS, RM1, AN4, HUB4
language model
Language Model
  • Word likelihoods
language model14
Language Model
  • ARPA format Example:

1-grams:

-3.7839 board -0.1552

-2.5998 bottom -0.3207

-3.7839 bunch -0.2174

2-grams:

-0.7782 as the -0.2717

-0.4771 at all 0.0000

-0.7782 at the -0.2915

3-grams:

-2.4450 in the lowest

-0.5211 in the middle

-2.4450 in the on

grammar
Grammar

public <basicCmd> = <startPolite> <command> <endPolite>;

public <startPolite> = (please | kindly | could you ) *;

public <endPolite> = [ please | thanks | thank you ];

<command> = <action> <object>;

<action> = (open | close | delete | move);

<object> = [the | a] (window | file | menu);

dictionary
Dictionary
  • Maps words to phoneme sequences
dictionary17
Dictionary
  • Example from cmudict.06d

POULTICE P OW L T AH S

POULTICES P OW L T AH S IH Z

POULTON P AW L T AH N

POULTRY P OW L T R IY

POUNCE P AW N S

POUNCED P AW N S T

POUNCEY P AW N S IY

POUNCING P AW N S IH NG

POUNCY P UW NG K IY

linguist
Linguist
  • Constructs the search graph of HMMs from:
    • Acoustic model
    • Statistical Language model ~or~
    • Grammar
    • Dictionary
search graph21
Search Graph
  • Can be statically or dynamically constructed
linguist types
Linguist Types
  • FlatLinguist
linguist types23
Linguist Types
  • FlatLinguist
  • DynamicFlatLinguist
linguist types24
Linguist Types
  • FlatLinguist
  • DynamicFlatLinguist
  • LexTreeLinguist
decoder
Decoder
  • Maps feature vectors to search graph
search manager
Search Manager
  • Searches the graph for the “best fit”
search manager27
Search Manager
  • Searches the graph for the “best fit”
  • P(sequence of feature vectors| word/phone)
  • aka. P(O|W)

-> “how likely is the input to have been generated by the word”

slide28

F ay ay ay ay v v v v v

F f ay ay ay ay v v v v

F f f ay ay ay ay v v v

F f f f ay ay ay ay v v

F f f f ay ay ay ay ay v

F f f f f ay ay ay ay v

F f f f f f ay ay ay v

pruner
Pruner
  • Uses algorithms to weed out low scoring paths during decoding
result
Result
  • Words!
word error rate
Word Error Rate
  • Most common metric
  • Measure the # of modifications to transform recognized sentence into reference sentence
word error rate33
Word Error Rate
  • Reference: “This is a reference sentence.”
  • Result: “This is neuroscience.”
word error rate34
Word Error Rate
  • Reference: “This is a reference sentence.”
  • Result: “This is neuroscience.”
  • Requires 2 deletions, 1 substitution
word error rate35
Word Error Rate
  • Reference: “This is a reference sentence.”
  • Result: “This is neuroscience.”
word error rate36
Word Error Rate
  • Reference: “This is a reference sentence.”
  • Result: “This is neuroscience.”
  • D S D
where speech recognition works
Where Speech Recognition Works
  • Limited Vocab Multi-Speaker
where speech recognition works49
Where Speech Recognition Works
  • Limited Vocab Multi-Speaker
  • Extensive Vocab Single Speaker
where speech recognition works50
Where Speech Recognition Works

*If you have noisy audio input multiply expected error rate x 2

where speech recognition works51
Where Speech Recognition Works

Other variables:

-Continuous vs. Isolated

-Conversational vs. Read

-Dialect

appendix i viterbi algorithm54
Appendix I: Viterbi Algorithm

P(ay | f) *

P(O2|ay)

P(f|f) *

P(O2 | f)

Time

O1

O2

O3

appendix i viterbi algorithm55
Appendix I: Viterbi Algorithm

P (O1) *

P(ay | f) *

P(O2|ay)

Time

O1

O2

O3

appendix ii faqs
Appendix II: FAQs
  • Common Sphinx4 FAQs can be found online:http://cmusphinx.sourceforge.net/sphinx4/doc/Sphinx4-faq.html
  • What followes are some less-FAQs
appendix ii faqs58
Appendix II: FAQs
  • Q. Is a search graph created for every recognition result or one for the recognition app?
  • A. This depends on which Linguist is used. The flat linguist generates the entire search graph and holds it in memory. It is only useful for small vocab recognition tasks. The lexTreeLinguist dynamically generates search states allowing it to handle very large vocabularies
appendix ii faqs59
Appendix II: FAQs
  • Q. How does the Viterbi algorithm save computation over exhaustive search?
  • A. The Viterbi algorithm saves memory and computation by reusing subproblems already solved within the larger solution. In this way probability calculations which repeat in different paths through the search graph do not get calculated multiple times
  • Viterbi cost = n2 – n3
  • Exhaustive search cost = 2n -3n
appendix ii faqs60
Appendix II: FAQs
  • Q. Does the linguist use a grammar to construct the search graph if it is available?
  • A. Yes, a grammar graph is created
appendix ii faqs61
Appendix II: FAQs
  • Q. What algorithm does the Pruner use?
  • A. Sphinx4 uses absolute and relative beam pruning
appendix iii configuration parameters
Appendix III: Configuration Parameters
  • Absolute Beam Width - # active search paths
  • <property name="absoluteBeamWidth" value="5000"/>
appendix iii configuration parameters63
Appendix III: Configuration Parameters
  • Absolute Beam Width - # active search paths
  • <property name="absoluteBeamWidth" value="5000"/>
  • Relative Beam Width – probability threshold
  • <property name="relativeBeamWidth" value="1E-120"/>
appendix iii configuration parameters64
Appendix III: Configuration Parameters
  • Absolute Beam Width - # active search paths
  • <property name="absoluteBeamWidth" value="5000"/>
  • Relative Beam Width – probability threshold
  • <property name="relativeBeamWidth" value="1E-120"/>
  • Word Insertion Probability – Word break likelihood
  • <property name="wordInsertionProbability" value="0.7"/>
appendix iii configuration parameters65
Appendix III: Configuration Parameters
  • Absolute Beam Width - # active search paths
  • <property name="absoluteBeamWidth" value="5000"/>
  • Relative Beam Width – probability threshold
  • <property name="relativeBeamWidth" value="1E-120"/>
  • Word Insertion Probability – Word break likelihood
  • <property name="wordInsertionProbability" value="0.7"/>
  • Language Weight – Boosts language model scores
  • <property name="languageWeight" value="10.5"/>
appendix iii configuration parameters66
Appendix III: Configuration Parameters
  • Silence Insertion Probability – Likelihood of inserting silence
  • <property name="silenceInsertionProbability" value=".1"/>
appendix iii configuration parameters67
Appendix III: Configuration Parameters
  • Silence Insertion Probability – Likelihood of inserting silence
  • <property name="silenceInsertionProbability" value=".1"/>
  • Filler Insertion Probability – Likelihood of inserting filler words
  • <property name="fillerInsertionProbability" value="1E-10"/>
appendix iv python note
Appendix IV: Python Note
  • To call a Java example from Python:

import subprocess

subprocess.call(["java", "-mx1000m", "-jar",

"/Users/Username/sphinx4/bin/Transcriber.jar”)

references
References
  • Speech and Language Processing 2nd Ed.Daniel Jurafsky and James MartinPearson, 2009
  • Artificial Intelligence 6th Ed.George LugerAddison Wesley, 2009
  • Sphinx Whitepaperhttp://cmusphinx.sourceforge.net/sphinx4/#whitepaper
  • Sphinx Forumhttps://sourceforge.net/projects/cmusphinx/forums