Current methods in forensic speaker ID: results of the fake case
This presentation is the property of its rightful owner.
Sponsored Links
1 / 31

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld Dutch Ministry of Justice formerly employed by the Netherlands Forensic Institute currently employed by the Immigration and Naturalisation Service IAFPA 24 July 2006. Outline. Main goal

Download Presentation

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Current methods in forensic speaker id results of the fake case tina cambier langeveld

Current methods in forensic speaker ID: results of the fake case

Tina Cambier-Langeveld

Dutch Ministry of Justice

formerly employed by the Netherlands Forensic Institute

currently employed by the Immigration and Naturalisation Service

IAFPA

24 July 2006


Outline

Outline

  • Main goal

  • Set-up: preparations, material, time path

  • The participants

  • Results summarized per file: Q1-10

  • Summary of the results

  • Other findings

  • Conclusion


Main goal

Main goal

  • To document the different methodologies and strategies that are used in different laboratories / by different experts to come to a final conclusion in a speaker verification task, based on an examination of the same set of materials

  • Have as many experts participate as possible (representative of the field, i.e. including all methods that are currently being used)

  • Not a proficiency test but a collaborative exercise


Preparations

Preparations

  • 20 experts were found willing to participate

  • Erik (a former colleague) asked three Canadian friends if we could record some telephone conversations with them for this purpose

  • The friend who could also be reached at work was picked as ‘suspect’ or Reference speaker

  • Jos Bouten acted as potential customer and speaking partner for the Reference recordings


Material audio

Material (audio)

  • The Questioned recordings (Q1-10) were telephone conversations between friends, very informal, varying in duration between 22 seconds and 11 minutes (file duration, i.e. including ringing)

  • The Reference recordings (R1-2) were telephone conversations with a potential customer asking about their services in detail, more formal, quite long: about 5 and 11 minutes (file duration), each containing > 2 minutes of net speech


Material questionnaire

Material (questionnaire)

A questionnaire was added, asking about the experience of the experts (in general and in English casework), the software that was used, the time spent on the fake case, any restrictions that the institute or the law imposes, comments, etc.


Time path

Time path

  • The materials were sent out to 20 potential participants in August-September 2004

  • Reports were due back before 1 February 2005

  • The final (12th) report was received on 29 April 2005, i.e. three months later

  • Definite results are presented to you now (as planned)

  • Paper for the International Journal of Speech, Language and the Law is currently under review


The participants 1

The participants (1)

  • 12 participants from 10 different countries, within and outside of Europe, employed by forensic laboratories, universities, police labs, the military etc. or working as private experts

    Methods:

  • 2 fully automatic

  • 5 auditory-phonetic, all members of IAFPA

  • 5 semi-automatic (variable)

    pretty representative of the field :-)


The participants 2

The participants (2)

  • Experience of the participants ranges between 28 years and one case; average >10 years

  • 5 of the 12 participants had never worked on a case in English before

  • 5 of the 12 participants had done between 1 and 3 cases in English before

  • 2 of the 12 participants had worked on cases in English a lot


Pooling of the results

Pooling of the results

The strength of the conclusion is not reported, only:

  • whether a conclusion was drawn

  • the direction of the conclusion (the same/not the same speaker as in the Reference recordings)

    …since the strength of the conclusion largely depends on the familiarity with the language, and so it would not be a fair comparison.


Exempted from the results

Exempted from the results

  • Two of the reports are excluded from this first section because the experts involved explicitly state that they have assumed that all of the ten questioned recordings came from the same source (1 semi-automatic, 1 auditory-phon.)

  • One of the reports is excluded from this section because the expert involved, after first ‘ensuring’ that all the questioned material is from one speaker (?), also examined the ten questioned recordings as a whole (1 auditory-phon.)


Results summarized per file

Results summarized per file…

...for 9 reports (2 x fully automatic, 4 x semi-automatic, 3 x auditory-phonetic)


Results summarized per file q1

Results summarized per file: Q1

-Net duration > 2 minutes

-notthe same speaker

8 x correct conclusion

1 x no decision (automatic analysis)


Results summarized per file q2

Results summarized per file: Q2

-Net duration: approx. 10 seconds

-the same speaker

3 x correct conclusion (2 x aud-phon, 1 x semi)

6 x too short / not enough information / no decision


Results summarized per file q3 4 6

Results summarized per file: Q3, 4, 6

-Net duration: more than 50 sec. each

-the same speaker

9 x correct conclusion


Results summarized per file q5

Results summarized per file: Q5

-Net duration: approx. 14 seconds

-the same speaker

7 x correct conclusion

2 x too short / not enough information (2 x semi)


Results summarized per file q7

Results summarized per file: Q7

-Net duration: approx. 5 seconds

-the same speaker

2 x correct conclusion (1 x aud-phon, 1 x semi)

7 x too short / not enough information / no decision


Results summarized per file q8

Results summarized per file: Q8

-Net duration: approx. 15 seconds

-notthe same speaker

6 x correct conclusion

3 x too short / not enough information / no decision (2 x semi, 1 x aut.)


Results summarized per file q9

Results summarized per file: Q9

-Net duration: approx. 14 seconds

-the same speaker

7 x correct conclusion

2 x too short / not enough information (2 x semi)


Results summarized per file q10

Results summarized per file: Q10

-Net duration: approx. 18 seconds

-notthe same speaker

2 x correct conclusion (1 x aut, 1 x semi)

5 x too short / not enough information / no decision

2 x wrong conclusion (1 x aud-phon, 1 x semi)


Summary of the results

Summary of the results

  • 3 participants (1 semi-automatic, 2 auditory-phonetic), assuming that all of the Q-files were from one speaker, failed to pick out the three conversations that were from different speakers

  • 2 other participants (1 semi-automatic, 1 auditory-phonetic) identify Q1 and Q8 as being from a different speaker, but incorrectly conclude that Q10 is from the Reference speaker.


Summary of the results1

Summary of the results

I have not been able to detect any obvious relation between the results and

  • the experience of the participants, in general or in English casework

  • their proficiency in English

  • the number of experts involved in the examination


Some other findings all 12 reports

Some other findings: all 12 reports

  • F0 measurements are done in 9 reports

  • Formant measurements are done in 7 reports

  • Speech rate / articulation rate / speaking rate measurements are done in 4 reports

    …but measurements are often not comparable, because they are implemented differently, done on different selections of the material or reported differently.


Some other findings

Some other findings

  • In 4 of the 5 auditory-phonetic reports, a difference in communicative context is noted

  • In 2 reports, the accent is correctly identified as Canadian, “possibly Ontario” / “maybe in the area of Toronto”

  • Documented feature ‘Canadian raising’ is referred to in 1 report (not a native of English)

  • Other accent remarks: RP, general American


Some other findings1

Some other findings

Gradations in the conclusion:

  • 4 x Bayesian format (Likelihood Ratio, or an amount of support for hypothesis H0 vs. H1)

  • 6 x in terms of likelihood, possibility etc. (all of these 6 scales are different! The number of possible conclusions ranges from 6 to 11…)

  • 1 x categorical (due to the legal system)

  • 1 x no gradations, just free text (private expert)


Questionnaire results

Questionnaire results

  • Time spent on the case ranges from 28 hours to 3 x 40 hours; average 65 hours

  • Time spent on an average case ranges from “8 hours for a 1:1 comparison” to “one month”. Average for the other 10 is 48 hours

  • Number of researchers involved: one (4x), two (3x), three (3x), four (2x), including technicians


Questionnaire results1

Questionnaire results

  • Software/speech analysis tools:

    Praat (5x), Cool Edit Pro/Adobe Audition (5x), SoundForge (3x), Kay CSL (3x), Wavesurfer (2x), SIVE (2x), Matlab (1x), x-waves Entropic (1x), DCLive Forensics (1x), own systems (5x)

    Some are typically suitable for editing and listening (Adobe Audition, SoundForge), others are better analysis tools (Praat, SIVE)


Questionnaire results2

Questionnaire results

  • “Is your analysis or report in any way restricted or enforced by e.g. the law, guidelines of your institute, accreditation boards, or previous court decisions?”

    - Four experts mention their laboratory / institute in their answer to this question

    - Only one participant mentioned the IAFPA Code of Conduct…….


Questionnaire results3

Questionnaire results

Comments include:

  • examination limited due to the language (2x)

  • examination limited for other reasons (3x)

  • took up too much time (2x)

  • “This was a very typical case for me”

  • “well chosen material, not too difficult, not too easy!”

  • words of appreciation (5x) ;-)


Questionnaire results4

Questionnaire results

Suggestions for the next exercise:

  • mismatched recording conditions (2x)

  • another language / various languages / multilingual speech (4x)

  • straight 1:1 comparison

  • different speaking styles


Conclusion

Conclusion

General comments:

  • Gives great insight into the different methods that are used

  • Reports vary widely on almost every aspect you can think of, and overlap is very limited, also between experts using the same method

  • Some of the reports may be made available (selectively), together with the audio material


  • Login