Contents

1. Whither Linguistic Interpretation of Acoustic Pronunciation VariationAnnika Hämäläinen, Yan Han, Lou Boves & Louis ten Bosch

2. Contents • Introduction • Objectives • Trajectory clustering: short introduction • Speech material • Evaluation of trajectory clustering: ASR • Phonetic and linguistic analysis • Relationship between trajectory clusters and transcription variants • Relationship between trajectory clusters and linguistic properties • Summary

3. Introduction (1/2) • Syllable-length acoustic models are expected to be better suited for modelling long-term spectral and temporal dependencies in speech • No need for precise segmental modelling • A large number of factors affect the way syllables are pronounced: • Phonetic context • Position in a multisyllabic word and in a sentence • Lexical stress and accent • Speaking rate • etc.

4. Introduction (2/2) • Because of the diverse sources of pronunciation variation, it may be necessary to create multi-path syllable models to capture variation that makes a difference for ASR performance. • Methods to alleviate the data sparsity problem (Sethy & Narayanan, 2003): • Combining syllable models for frequent syllables with triphones covering the less frequent syllables • Bootstrapping the topologies and observation densities of the syllable models using triphones

5. Objectives • To study trajectory clustering as a method of building multi-path syllable models. • To investigate whether there is a relationship between phonetic/linguistic properties and the results of trajectory clustering. • Such a relationship could be utilised in building or adapting multi-path syllable models.

6. Trajectory Clustering (Han et al., 2005) • Deriving homogeneous clusters of longer-length models directly from the speech signal: • Sound intervals regarded as continuous trajectories along time in observation space • Sound intervals clustered based on the similarity of the trajectories • An individual path created for each cluster • Parallel paths used during recognition

7. Speech Material • Female read speech from the Spoken Dutch Corpus

8. Evaluation of Trajectory Clustering: ASR

9. Speech Recognition / Method • Baseline: Triphone recogniser • Experimental recognisers: • Syllable models for 94 most frequent syllables; triphones used to cover the rest of the syllables • The path topologies and observation densities of syllable models bootstrapped using triphones corresponding to canonical syllable transcriptions and trained further using Baum-Welch re-estimation • 1-path mixed-model recogniser • All tokens of a given syllable used for training the single path • 2-path & 3-path mixed-model recognisers • Trajectory clustering used to divide the syllable tokens for training the parallel paths

10. Speech Recognition / Results & Conclusions • Single path not sufficient to capture syllable-level variation • 2-path syllable models capture important pronunciation variation and lead to improved recognition performance • Undertraining of the 3-path syllable models hindering performance

11. Phonetic Analysis

12. Phonetic Analysis / Method • To check whether syllable tokens with different phonetic transcriptions go into different clusters: • Phonetic distances between the pronunciation variants of each syllable were computed on the basis of articulatory features • A multidimensional scaling (MDS) analysis was carried out for 1- or 2-dimensional representations of the phonetic distances between the pronunciation variants • The MDS distance representations were compared with the clusters produced by trajectory clustering

13. Example: syllable /O_f/ Phonetic Analysis / Results Proportions of pronunciation variant tokens assigned to clusters 2-dimensional MDS distance representation

14. Even though MDS produced phonetically solid distance representations, it appeared that there was no clear correspondence between the clusters of syllable transcription variants produced by the MDS analysis and the clusters produced by trajectory clustering. • Further analysis needed, as the varying numbers of tokens in the different clusters makes the interpretation of the results difficult. Phonetic Analysis / Conclusions

15. Linguistic Analysis

16. Linguistic Analysis / Method • To check whether syllable tokens with certain linguistic properties go into different clusters, a graphical representation was used to compare the 2-way clusters produced by trajectory clustering with 2-way clusters based on the following linguistic properties: • Duration (long vs. short syllable) • POS (function vs. content word) • Lexical stress (stressed vs. unstressed syllable) • Monosyllabicity (mono-syllabic vs. multisyllabic word)

17. Linguistic Analysis / Results (2/2) • Overall pattern:

18. There were hardly any syllables showing a systematic connection between the linguistic properties tested and the results of trajectory clustering. Linguistic Analysis / Conclusions

19. Summary • Improved ASR performance suggests that trajectory clustering is an attractive way of building multi-path syllable models • There is no straightforward relationship between the acoustically defined clusters and the phonetic/linguistic factors tested in this study.  Designing or adapting multi-path syllable models based on such properties seems very difficult.

20. Questions?

21. Example syllables: /t_ei_t/, /z_o/, /l_@/ and /h_a_r/ Linguistic Analysis / Results (1/2)