slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Current Research in Forensic Toolmark Analysis PowerPoint Presentation
Download Presentation
Current Research in Forensic Toolmark Analysis

Loading in 2 Seconds...

play fullscreen
1 / 23

Current Research in Forensic Toolmark Analysis - PowerPoint PPT Presentation

  • Uploaded on

Current Research in Forensic Toolmark Analysis Helping to satisfy the “new” needs of forensic scientists with state of the art microscopy, computation and statistics. Outline. Introduction Instruments for 3D toolmark analysis 3D t oolmark data The statistics:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Current Research in Forensic Toolmark Analysis' - molimo

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Current Research in Forensic

Toolmark Analysis

Helping to satisfy the “new” needs of forensic scientists with state of the art microscopy, computation and statistics



  • Introduction
  • Instruments for 3D toolmark analysis
  • 3D toolmark data
  • The statistics:
    • Identification Error Rates
    • “Match” confidence
    • “Match” probability
  • Statistics from available practitioner data
quantitative criminalistics
Quantitative Criminalistics
  • All forms of physical evidence can be represented asnumerical patterns
    • Toolmark surfaces
    • Dust and soil categories and spectra
    • Hair/Fiber categories and spectra
    • Craniofacial landmarks
    • Triangulated fingerprint minutiae
  • Machine learning trains a computer to recognize patterns
    • Can give “…the quantitative difference between an identification and non-identification”Moran
    • Can yield identification error rate estimates
    • May be even confidence measures for I.D.s

Data Acquisition For Toolmarks

Confocal Microscope

Scanning Electron Microscope

Focus Variation Microscope

Comparison Microscope


Screwdriver Striation Patterns in Lead

3D surfaces


2D profiles



Bullet base, 9mm Ruger Barrel

Bullet base, 9mm Glock Barrel


What can we do with all this microscope data?

  • Statistical pattern comparison!
  • Modern algorithms are called machine learning
  • Idea is to measure features that characterize physical evidence
  • Train algorithm to recognize “major” differences between groups of features

while taking into account

natural variation and measurement error.


Visually explore: 3D PCA of 760 real and simulated mean profiles of primer shears from 24 Glocks:

  • ~45% variance retained

Support Vector Machines

  • Support Vector Machines (SVM) determine efficient association rules
    • In the absence of specific knowledge of probability densities

SVM decision boundary


18D PCA-SVM Primer Shear I.D. Model, 2000 Bootstrap Resamples

Refined bootstrapped I.D. error rate for primer shear striation patterns= 0.35%

95% C.I. = [0%, 0.83%]

(sample size = 720 real and simulated profiles)


How good of a “match” is it?

Conformal PredictionVovk

  • Can give a judge or jury an easy to understand measure of reliability of classification result
  • Confidence on a scale of 0%-100%
  • Testable claim: Long run I.D. error-rate should be the chosen significance level

80% confidence

20% error

Slope = 0.2

99% confidence

1% error

Slope = 0.01

95% confidence

5% error

Slope = 0.05

  • This is an orthodox “frequentist”
  • approach
      • Roots in Algorithmic Information Theory

Cumulative # of Errors

  • Data should be IID but that’s it

Sequence of Unk Obs Vects


Conformal Prediction

  • For 95%-CPT (PCA-SVM) confidence intervals will not contain the correct I.D. 5% of the time in the long run
    • Straight-forward validation/explanation picture for court

Empirical Error Rate: 5.3%

Theoretical (Long Run)

Error Rate: 5%

14D PCA-SVM Decision Model

for screwdriver striation patterns


How good of a “match” is it?

Efron Empirical Bayes’

  • An I.D. is output for each questioned toolmark
    • This is a computer “match”
  • What’s the probability the tool is truly the source of the toolmark?
  • Similar problem in genomics for detecting disease from microarray data
    • They use dataandBayes’ theorem to get an estimate

A Bayesian Hierarchical Model: Believability Curve

JAGS MCMC Bayesian over-dispersed Poisson with intercept, on test set


Bayes Factors/Likelihood Ratios

  • In the “Forensic Bayesian Framework”, the Likelihood Ratio is the measure of the weight of evidence.
    • LRs are called Bayes Factors by most statistician
    • LRs give the measure of support the “evidence” lends to the “prosecution hypothesis” vs. the “defense hypothesis”
  • From Bayes Theorem:

Bayes Factors/Likelihood Ratios

  • Using the fit posteriors and priors we can obtain the likelihood ratiosTippett, Ramos

Known match LR values

Known non-match LR values


Available Large Scale Practitioner Studies

  • Two large scale published studies
    • 10-Barrel TestHamby:
      • 626 practitioners (24 countries)
      • 15 “unknowns” per test set
      • At least one bullet from each of the 10 consecutively manufactured barrels
      • # examiner errors committed = 0
    • GLOCK Cartridge Case TestHamby:
      • 1632 9-mm Glock fired cartridge cases
      • 1 case per Glock
      • All cartridge cases pair-wise compared
      • # of pairs of cartridge cases judged to have enough surface detail agreement to be (falsely) “matching” = 0
        • AFTE Theory of Identification standard used:

Available Large Scale Practitioner Studies

  • 0% error rate is the “frequentist” estimate
    • We looked to sports statistics for low scoring games
    • “Bayesian” statistics provide complementary methods for analysis
      • Can work much better in estimating small probabilities

So does that mean the error rate is 0%?


Available Large Scale Practitioner Studies

  • For 10-Barrel we need to estimate a small error rate
  • For GLOCK we need to estimate a small random match probability (RMP)
  • Use Bayesian “Beta-binomial” method when no “failures” are observed (Schuckers)

Available Large Scale Practitioner Studies

  • Basic idea of the reverend Bayes:

Updated Knowledge

Prior Knowledge × Data =


a + b

Error Rate/RMP =

Uninf(a,b) × Beta-Binomial(data | a,b)

Posterior(a,b | data)

Get updated estimates of Error rate/RMP


Available Large Scale Practitioner Studies

  • So given the observed data and assuming “prior ignorance”
    • Posterior error rate/RMP distributions:

Posterior Dist. 10-Barrel

Posterior Dist. GLOCK



[0.0000020%, 0.00031%]

Average Examiner Error Rate


[0.00023%, 0.040%]

  • Professor Chris Saunders (SDSU)
  • Professor Christophe Champod (Lausanne)
  • Alan Zheng (NIST)
  • Research Team:
    • Dr. Martin Baiker
    • Ms. Helen Chan
    • Ms. Julie Cohen
    • Mr. Peter Diaczuk
    • Dr. Peter De Forest
    • Mr. Antonio Del Valle
    • Ms. Carol Gambino
    • Dr. James Hamby
  • Ms. Alison Hartwell, Esq.
  • Dr. Thomas Kubic, Esq.
  • Ms. Loretta Kuo
  • Ms. Frani Kammerman
  • Dr. Brooke Kammrath
  • Mr. Chris Lucky
  • Off. Patrick McLaughlin
  • Dr. Linton Mohammed
  • Mr. Nicholas Petraco
  • Dr. Dale Purcel
  • Ms. Stephanie Pollut
  • Dr. Peter Pizzola
  • Dr. Graham Rankin
  • Dr. Jacqueline Speir
  • Dr. Peter Shenkin
  • Ms. Rebecca Smith
  • Mr. Chris Singh
  • Mr. Peter Tytell
  • Ms. Elizabeth Willie
  • Ms. Melodie Yu
  • Dr. Peter Zoon