Current methods in forensic speaker ID: results of the fake case
Download
1 / 31

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld - PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld Dutch Ministry of Justice formerly employed by the Netherlands Forensic Institute currently employed by the Immigration and Naturalisation Service IAFPA 24 July 2006. Outline. Main goal

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld' - yetta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Current methods in forensic speaker ID: results of the fake case

Tina Cambier-Langeveld

Dutch Ministry of Justice

formerly employed by the Netherlands Forensic Institute

currently employed by the Immigration and Naturalisation Service

IAFPA

24 July 2006


Outline
Outline case

  • Main goal

  • Set-up: preparations, material, time path

  • The participants

  • Results summarized per file: Q1-10

  • Summary of the results

  • Other findings

  • Conclusion


Main goal
Main goal case

  • To document the different methodologies and strategies that are used in different laboratories / by different experts to come to a final conclusion in a speaker verification task, based on an examination of the same set of materials

  • Have as many experts participate as possible (representative of the field, i.e. including all methods that are currently being used)

  • Not a proficiency test but a collaborative exercise


Preparations
Preparations case

  • 20 experts were found willing to participate

  • Erik (a former colleague) asked three Canadian friends if we could record some telephone conversations with them for this purpose

  • The friend who could also be reached at work was picked as ‘suspect’ or Reference speaker

  • Jos Bouten acted as potential customer and speaking partner for the Reference recordings


Material audio
Material (audio) case

  • The Questioned recordings (Q1-10) were telephone conversations between friends, very informal, varying in duration between 22 seconds and 11 minutes (file duration, i.e. including ringing)

  • The Reference recordings (R1-2) were telephone conversations with a potential customer asking about their services in detail, more formal, quite long: about 5 and 11 minutes (file duration), each containing > 2 minutes of net speech


Material questionnaire
Material (questionnaire) case

A questionnaire was added, asking about the experience of the experts (in general and in English casework), the software that was used, the time spent on the fake case, any restrictions that the institute or the law imposes, comments, etc.


Time path
Time path case

  • The materials were sent out to 20 potential participants in August-September 2004

  • Reports were due back before 1 February 2005

  • The final (12th) report was received on 29 April 2005, i.e. three months later

  • Definite results are presented to you now (as planned)

  • Paper for the International Journal of Speech, Language and the Law is currently under review


The participants 1
The participants (1) case

  • 12 participants from 10 different countries, within and outside of Europe, employed by forensic laboratories, universities, police labs, the military etc. or working as private experts

    Methods:

  • 2 fully automatic

  • 5 auditory-phonetic, all members of IAFPA

  • 5 semi-automatic (variable)

    pretty representative of the field :-)


The participants 2
The participants (2) case

  • Experience of the participants ranges between 28 years and one case; average >10 years

  • 5 of the 12 participants had never worked on a case in English before

  • 5 of the 12 participants had done between 1 and 3 cases in English before

  • 2 of the 12 participants had worked on cases in English a lot


Pooling of the results
Pooling of the results case

The strength of the conclusion is not reported, only:

  • whether a conclusion was drawn

  • the direction of the conclusion (the same/not the same speaker as in the Reference recordings)

    …since the strength of the conclusion largely depends on the familiarity with the language, and so it would not be a fair comparison.


Exempted from the results
Exempted from the results case

  • Two of the reports are excluded from this first section because the experts involved explicitly state that they have assumed that all of the ten questioned recordings came from the same source (1 semi-automatic, 1 auditory-phon.)

  • One of the reports is excluded from this section because the expert involved, after first ‘ensuring’ that all the questioned material is from one speaker (?), also examined the ten questioned recordings as a whole (1 auditory-phon.)


Results summarized per file
Results summarized per file… case

...for 9 reports (2 x fully automatic, 4 x semi-automatic, 3 x auditory-phonetic)


Results summarized per file q1
Results summarized per file: Q1 case

-Net duration > 2 minutes

-notthe same speaker

8 x correct conclusion

1 x no decision (automatic analysis)


Results summarized per file q2
Results summarized per file: Q2 case

-Net duration: approx. 10 seconds

-the same speaker

3 x correct conclusion (2 x aud-phon, 1 x semi)

6 x too short / not enough information / no decision


Results summarized per file q3 4 6
Results summarized per file: Q3, 4, 6 case

-Net duration: more than 50 sec. each

-the same speaker

9 x correct conclusion


Results summarized per file q5
Results summarized per file: Q5 case

-Net duration: approx. 14 seconds

-the same speaker

7 x correct conclusion

2 x too short / not enough information (2 x semi)


Results summarized per file q7
Results summarized per file: Q7 case

-Net duration: approx. 5 seconds

-the same speaker

2 x correct conclusion (1 x aud-phon, 1 x semi)

7 x too short / not enough information / no decision


Results summarized per file q8
Results summarized per file: Q8 case

-Net duration: approx. 15 seconds

-notthe same speaker

6 x correct conclusion

3 x too short / not enough information / no decision (2 x semi, 1 x aut.)


Results summarized per file q9
Results summarized per file: Q9 case

-Net duration: approx. 14 seconds

-the same speaker

7 x correct conclusion

2 x too short / not enough information (2 x semi)


Results summarized per file q10
Results summarized per file: Q10 case

-Net duration: approx. 18 seconds

-notthe same speaker

2 x correct conclusion (1 x aut, 1 x semi)

5 x too short / not enough information / no decision

2 x wrong conclusion (1 x aud-phon, 1 x semi)


Summary of the results
Summary of the results case

  • 3 participants (1 semi-automatic, 2 auditory-phonetic), assuming that all of the Q-files were from one speaker, failed to pick out the three conversations that were from different speakers

  • 2 other participants (1 semi-automatic, 1 auditory-phonetic) identify Q1 and Q8 as being from a different speaker, but incorrectly conclude that Q10 is from the Reference speaker.


Summary of the results1
Summary of the results case

I have not been able to detect any obvious relation between the results and

  • the experience of the participants, in general or in English casework

  • their proficiency in English

  • the number of experts involved in the examination


Some other findings all 12 reports
Some other findings: all 12 reports case

  • F0 measurements are done in 9 reports

  • Formant measurements are done in 7 reports

  • Speech rate / articulation rate / speaking rate measurements are done in 4 reports

    …but measurements are often not comparable, because they are implemented differently, done on different selections of the material or reported differently.


Some other findings
Some other findings case

  • In 4 of the 5 auditory-phonetic reports, a difference in communicative context is noted

  • In 2 reports, the accent is correctly identified as Canadian, “possibly Ontario” / “maybe in the area of Toronto”

  • Documented feature ‘Canadian raising’ is referred to in 1 report (not a native of English)

  • Other accent remarks: RP, general American


Some other findings1
Some other findings case

Gradations in the conclusion:

  • 4 x Bayesian format (Likelihood Ratio, or an amount of support for hypothesis H0 vs. H1)

  • 6 x in terms of likelihood, possibility etc. (all of these 6 scales are different! The number of possible conclusions ranges from 6 to 11…)

  • 1 x categorical (due to the legal system)

  • 1 x no gradations, just free text (private expert)


Questionnaire results
Questionnaire results case

  • Time spent on the case ranges from 28 hours to 3 x 40 hours; average 65 hours

  • Time spent on an average case ranges from “8 hours for a 1:1 comparison” to “one month”. Average for the other 10 is 48 hours

  • Number of researchers involved: one (4x), two (3x), three (3x), four (2x), including technicians


Questionnaire results1
Questionnaire results case

  • Software/speech analysis tools:

    Praat (5x), Cool Edit Pro/Adobe Audition (5x), SoundForge (3x), Kay CSL (3x), Wavesurfer (2x), SIVE (2x), Matlab (1x), x-waves Entropic (1x), DCLive Forensics (1x), own systems (5x)

    Some are typically suitable for editing and listening (Adobe Audition, SoundForge), others are better analysis tools (Praat, SIVE)


Questionnaire results2
Questionnaire results case

  • “Is your analysis or report in any way restricted or enforced by e.g. the law, guidelines of your institute, accreditation boards, or previous court decisions?”

    - Four experts mention their laboratory / institute in their answer to this question

    - Only one participant mentioned the IAFPA Code of Conduct…….


Questionnaire results3
Questionnaire results case

Comments include:

  • examination limited due to the language (2x)

  • examination limited for other reasons (3x)

  • took up too much time (2x)

  • “This was a very typical case for me”

  • “well chosen material, not too difficult, not too easy!”

  • words of appreciation (5x) ;-)


Questionnaire results4
Questionnaire results case

Suggestions for the next exercise:

  • mismatched recording conditions (2x)

  • another language / various languages / multilingual speech (4x)

  • straight 1:1 comparison

  • different speaking styles


Conclusion
Conclusion case

General comments:

  • Gives great insight into the different methods that are used

  • Reports vary widely on almost every aspect you can think of, and overlap is very limited, also between experts using the same method

  • Some of the reports may be made available (selectively), together with the audio material


ad