1 / 6

Harmonic-Temporal Clustering of Speech

This study explores a novel approach for robust F0 analysis in complex acoustic environments, enabling accurate pitch estimation in speech recognition and enhancement applications. The method combines harmonic modeling and temporal clustering to extract pitch contours in the presence of various noise sources. The proposed algorithm utilizes parametric models and Gaussian mixture models to characterize both voiced speech and background noise components, allowing for accurate pitch estimation even in challenging scenarios. Experimental results demonstrate the effectiveness of the approach in accurately capturing multiple pitch contours in co-channel speech settings.

tallulah-ramos
Download Presentation

Harmonic-Temporal Clustering of Speech

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Harmonic-Temporal Clustering of Speech Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Alain de Cheveigné, Shigeki Sagayama

  2. Motivation and Approach • Precise and Robust F0 analysis • Analysis of complex and varied acoustical scenes • For speech, applications in speech recognition, prosody analysis, speech enhancement, speaker identification… • Desirable features of a new pitch determination algorithm (PDA) • The performance should stay high in a wide range of background noises (white noise, pink noise, noise bursts, music, other speech) • Extracting simultaneously the pitch contours of several concurrent voices is possible • Overall speech model, spectro-temporal model with constraints • Several existing multi-pitch tracking algorithms: initial frame-by-frame analysis, then post-processing to reduce errors and obtain a smooth pitch contour (for example using HMMs) • We propose to perform estimation and model-based interpolation simultaneously: • Parametric model of the voiced parts of the power spectrum of speech • Introduction of a noise model to extract harmonically structured “islands” within a “sea” of unstructured noise.

  3. Overview of the method • Express the whole pitch contour as a smooth curve→ cubic spline • Distribute audio objects with different acoustical properties • Express the harmonic structure as a parametric function: GMM • Express the power envelope in time direction as a parametric function: GMM • Characteristic: • Through the harmonicity assumption, the method models the voiced parts of speech Log-Frequency Simultaneous optimization of the parameters time

  4. F0 estimation in noisy environments • Speech mixed with broadband background noise: • Voiced speech with several types of interferences: Accuracy (%) of the F0 estimation:

  5. Multi-pitch estimation • Co-channel speech of two speakers speaking simultaneously with equal average power. • Test data • Bagshaw database、１５０ mixtures • 16kHz, monaural signal • Results 8kHz Frequency 50Hz 0s time 1.3s 1.3s 0s 「a-o-i」 「o-i-o-o-u」 No second sound here

More Related