New Methods for Voice Quality Evaluation for IP Networks - PowerPoint PPT Presentation

new methods for voice quality evaluation for ip networks n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
New Methods for Voice Quality Evaluation for IP Networks PowerPoint Presentation
Download Presentation
New Methods for Voice Quality Evaluation for IP Networks

play fullscreen
1 / 21
New Methods for Voice Quality Evaluation for IP Networks
129 Views
Download Presentation
bree-gilbert
Download Presentation

New Methods for Voice Quality Evaluation for IP Networks

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. New Methods for Voice Quality Evaluation for IP Networks

  2. Outline • Introduction • Subjective MOS test from traditional to Internet based • Intrusive voice quality measurement from listening-only to conversational • Non-intrusive measurement from subjective based to objective based • Conclusions ITC-18, Berlin, Germany

  3. Introduction • Aims: to investigate new subjective and objective measurement methods for VoIP applications • Subjective tests • MOS (Mean Opinion Score), user perceived quality • Benchmarking objective methods • Slow, time consuming and expensive • Objective measurements • Intrusive methods (e.g. PESQ, only listening quality) • Non-intrusive methods (e.g. E-model, only for limited codec/applications) ITC-18, Berlin, Germany

  4. Traditional MOS test • Strict test requirement (e.g. sound proof room) • Guarantee consistent testing environment • Slow, time-consuming and expensive • Test environment far to reality • Originated from codec quality assessment (the quality difference is subtle). ITC-18, Berlin, Germany

  5. Internet-based MOS test • Based on Internet, carried out at office, project room or classroom, …, close to reality • Easier access to large number of subjects (e.g. 39 tests at the same time) • Save time and money • Lack of a controlled testing environment (e.g. background noise). ITC-18, Berlin, Germany

  6. Online MOS test ITC-18, Berlin, Germany

  7. Online MOS test ITC-18, Berlin, Germany

  8. Online MOS test results ITC-18, Berlin, Germany

  9. MOS (PESQ) PESQ Jitter buffer decoder encoder Trace data Degraded speech MOS (Web or Room) Subjective test E-model MOS (E-model) Voice Quality Evaluation Reference speech ITC-18, Berlin, Germany

  10. Performance Comparison ITC-18, Berlin, Germany

  11. Performance Comparison Correlation coefficients for MOS comparison • Results show that Internet-based MOS test compares well with traditional MOS test. • Two objective test methods (PESQ and E-model) can both predict subjective MOS score well. ITC-18, Berlin, Germany

  12. Intrusive conversational quality measurement • PESQ is the latest ITU standard for intrusive voice quality measurement. • It can only predict one-way listening voice quality. • How to extend it to predict conversational quality? • One solution: to combine with E-model ITC-18, Berlin, Germany

  13. MOS (PESQ) Reference speech Ie PESQ MOS  R  Ie E-model MOSc Degraded speech Delay model End-to-end delay Id Intrusive conversationalmeasurement • Procedures: • Obtain MOS from PESQ based on a comparison of reference and degraded speech • Convert it back to R-factor, then to Ie • Obtain Id from end-to-end delay • Calculate MOSc from Id and Ie from E-model ITC-18, Berlin, Germany

  14. R-factor with MOS ( From R to MOS, from G.107) (1) ( From MOS to R, by 3rd order polynomial) (2) ITC-18, Berlin, Germany

  15. Combined with E-model (3) (4) (5) Convert R back to MOS to obtain conversational voice quality ITC-18, Berlin, Germany

  16. Non-intrusiveobjective measurement • E-model is the latest ITU standard for non-intrusive voice quality prediction. • Its parameters/equations (e.g. Ie) are still based on subjective tests. • Difficult to apply to new codecs/applications • How to improve it ? • Based on objective methods (e.g. PESQ) to derive parameters/equations for new codecs and applications. ITC-18, Berlin, Germany

  17. Procedure 1:obtain MOS (PESQ) • Obtain MOS (PESQ) vs. packet loss rate (random) for the codec (e.g. AMR 12.2 Kb/s) • MOS (PESQ) value was obtained by averaging over different speech samples (male and female) in ITU data base and different loss locations/patterns. ITC-18, Berlin, Germany

  18. Procedure 2:Obtain Ie equation • Convert MOS vs. packet loss to Ie vs. packet loss using Equations (2) and (3). • Curve fitting to obtain equation of Ie vs. loss. (6) ITC-18, Berlin, Germany

  19. Procedure 3: Obtain MOS (loss, delay) • From Ie and Id, R can be derived, then MOS can be obtained directly from packet loss and end-to-end delay. • This can be used for non-intrusive quality monitoring and perceptual buffer optimization and perceived QoS control. ITC-18, Berlin, Germany

  20. Conclusions/Future work • Investigated novel subjective and objective voice quality evaluation methods. • Future work • More extensive Internet-based MOS test and comparison with P.800 MOS test • New applications for predicted perceived voice quality. ITC-18, Berlin, Germany

  21. Contact • MOS Test Website: http://www.tech.plymouth.ac.uk/spmc/people/lfsun/mos/ • Contact • Lingfen Sun: L.Sun@plymouth.ac.uk • Emmanuel Ifeachor: E.Ifeachor@plymouth.ac.uk Thank you! ITC-18, Berlin, Germany