1 / 31

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld Dutch Ministry of Justice formerly employed by the Netherlands Forensic Institute currently employed by the Immigration and Naturalisation Service IAFPA 24 July 2006. Outline. Main goal

yetta
Download Presentation

Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Current methods in forensic speaker ID: results of the fake case Tina Cambier-Langeveld Dutch Ministry of Justice formerly employed by the Netherlands Forensic Institute currently employed by the Immigration and Naturalisation Service IAFPA 24 July 2006

  2. Outline • Main goal • Set-up: preparations, material, time path • The participants • Results summarized per file: Q1-10 • Summary of the results • Other findings • Conclusion

  3. Main goal • To document the different methodologies and strategies that are used in different laboratories / by different experts to come to a final conclusion in a speaker verification task, based on an examination of the same set of materials • Have as many experts participate as possible (representative of the field, i.e. including all methods that are currently being used) • Not a proficiency test but a collaborative exercise

  4. Preparations • 20 experts were found willing to participate • Erik (a former colleague) asked three Canadian friends if we could record some telephone conversations with them for this purpose • The friend who could also be reached at work was picked as ‘suspect’ or Reference speaker • Jos Bouten acted as potential customer and speaking partner for the Reference recordings

  5. Material (audio) • The Questioned recordings (Q1-10) were telephone conversations between friends, very informal, varying in duration between 22 seconds and 11 minutes (file duration, i.e. including ringing) • The Reference recordings (R1-2) were telephone conversations with a potential customer asking about their services in detail, more formal, quite long: about 5 and 11 minutes (file duration), each containing > 2 minutes of net speech

  6. Material (questionnaire) A questionnaire was added, asking about the experience of the experts (in general and in English casework), the software that was used, the time spent on the fake case, any restrictions that the institute or the law imposes, comments, etc.

  7. Time path • The materials were sent out to 20 potential participants in August-September 2004 • Reports were due back before 1 February 2005 • The final (12th) report was received on 29 April 2005, i.e. three months later • Definite results are presented to you now (as planned) • Paper for the International Journal of Speech, Language and the Law is currently under review

  8. The participants (1) • 12 participants from 10 different countries, within and outside of Europe, employed by forensic laboratories, universities, police labs, the military etc. or working as private experts Methods: • 2 fully automatic • 5 auditory-phonetic, all members of IAFPA • 5 semi-automatic (variable) pretty representative of the field :-)

  9. The participants (2) • Experience of the participants ranges between 28 years and one case; average >10 years • 5 of the 12 participants had never worked on a case in English before • 5 of the 12 participants had done between 1 and 3 cases in English before • 2 of the 12 participants had worked on cases in English a lot

  10. Pooling of the results The strength of the conclusion is not reported, only: • whether a conclusion was drawn • the direction of the conclusion (the same/not the same speaker as in the Reference recordings) …since the strength of the conclusion largely depends on the familiarity with the language, and so it would not be a fair comparison.

  11. Exempted from the results • Two of the reports are excluded from this first section because the experts involved explicitly state that they have assumed that all of the ten questioned recordings came from the same source (1 semi-automatic, 1 auditory-phon.) • One of the reports is excluded from this section because the expert involved, after first ‘ensuring’ that all the questioned material is from one speaker (?), also examined the ten questioned recordings as a whole (1 auditory-phon.)

  12. Results summarized per file… ...for 9 reports (2 x fully automatic, 4 x semi-automatic, 3 x auditory-phonetic)

  13. Results summarized per file: Q1 -Net duration > 2 minutes -notthe same speaker 8 x correct conclusion 1 x no decision (automatic analysis)

  14. Results summarized per file: Q2 -Net duration: approx. 10 seconds -the same speaker 3 x correct conclusion (2 x aud-phon, 1 x semi) 6 x too short / not enough information / no decision

  15. Results summarized per file: Q3, 4, 6 -Net duration: more than 50 sec. each -the same speaker 9 x correct conclusion

  16. Results summarized per file: Q5 -Net duration: approx. 14 seconds -the same speaker 7 x correct conclusion 2 x too short / not enough information (2 x semi)

  17. Results summarized per file: Q7 -Net duration: approx. 5 seconds -the same speaker 2 x correct conclusion (1 x aud-phon, 1 x semi) 7 x too short / not enough information / no decision

  18. Results summarized per file: Q8 -Net duration: approx. 15 seconds -notthe same speaker 6 x correct conclusion 3 x too short / not enough information / no decision (2 x semi, 1 x aut.)

  19. Results summarized per file: Q9 -Net duration: approx. 14 seconds -the same speaker 7 x correct conclusion 2 x too short / not enough information (2 x semi)

  20. Results summarized per file: Q10 -Net duration: approx. 18 seconds -notthe same speaker 2 x correct conclusion (1 x aut, 1 x semi) 5 x too short / not enough information / no decision 2 x wrong conclusion (1 x aud-phon, 1 x semi)

  21. Summary of the results • 3 participants (1 semi-automatic, 2 auditory-phonetic), assuming that all of the Q-files were from one speaker, failed to pick out the three conversations that were from different speakers • 2 other participants (1 semi-automatic, 1 auditory-phonetic) identify Q1 and Q8 as being from a different speaker, but incorrectly conclude that Q10 is from the Reference speaker.

  22. Summary of the results I have not been able to detect any obvious relation between the results and • the experience of the participants, in general or in English casework • their proficiency in English • the number of experts involved in the examination

  23. Some other findings: all 12 reports • F0 measurements are done in 9 reports • Formant measurements are done in 7 reports • Speech rate / articulation rate / speaking rate measurements are done in 4 reports …but measurements are often not comparable, because they are implemented differently, done on different selections of the material or reported differently.

  24. Some other findings • In 4 of the 5 auditory-phonetic reports, a difference in communicative context is noted • In 2 reports, the accent is correctly identified as Canadian, “possibly Ontario” / “maybe in the area of Toronto” • Documented feature ‘Canadian raising’ is referred to in 1 report (not a native of English) • Other accent remarks: RP, general American

  25. Some other findings Gradations in the conclusion: • 4 x Bayesian format (Likelihood Ratio, or an amount of support for hypothesis H0 vs. H1) • 6 x in terms of likelihood, possibility etc. (all of these 6 scales are different! The number of possible conclusions ranges from 6 to 11…) • 1 x categorical (due to the legal system) • 1 x no gradations, just free text (private expert)

  26. Questionnaire results • Time spent on the case ranges from 28 hours to 3 x 40 hours; average 65 hours • Time spent on an average case ranges from “8 hours for a 1:1 comparison” to “one month”. Average for the other 10 is 48 hours • Number of researchers involved: one (4x), two (3x), three (3x), four (2x), including technicians

  27. Questionnaire results • Software/speech analysis tools: Praat (5x), Cool Edit Pro/Adobe Audition (5x), SoundForge (3x), Kay CSL (3x), Wavesurfer (2x), SIVE (2x), Matlab (1x), x-waves Entropic (1x), DCLive Forensics (1x), own systems (5x) Some are typically suitable for editing and listening (Adobe Audition, SoundForge), others are better analysis tools (Praat, SIVE)

  28. Questionnaire results • “Is your analysis or report in any way restricted or enforced by e.g. the law, guidelines of your institute, accreditation boards, or previous court decisions?” - Four experts mention their laboratory / institute in their answer to this question - Only one participant mentioned the IAFPA Code of Conduct…….

  29. Questionnaire results Comments include: • examination limited due to the language (2x) • examination limited for other reasons (3x) • took up too much time (2x) • “This was a very typical case for me” • “well chosen material, not too difficult, not too easy!” • words of appreciation (5x) ;-)

  30. Questionnaire results Suggestions for the next exercise: • mismatched recording conditions (2x) • another language / various languages / multilingual speech (4x) • straight 1:1 comparison • different speaking styles

  31. Conclusion General comments: • Gives great insight into the different methods that are used • Reports vary widely on almost every aspect you can think of, and overlap is very limited, also between experts using the same method • Some of the reports may be made available (selectively), together with the audio material

More Related