Slide1 l.jpg
Advertisement
This presentation is the property of its rightful owner.
1 / 36

Ballistics DNA PowerPoint PPT Presentation

Ballistics DNA Alain Beauchamp, PH.D. The path to a ballistic probability model PART I: Correlation score and probability PART II: Ballistic probability model PART III: How could we implement a probability model in a ballistic system? Conclusion and future work

Download Presentation

Ballistics DNA

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide2 l.jpg

Ballistics DNA

Alain Beauchamp, PH.D.


The path to a ballistic probability model l.jpg

The path to a ballistic probability model

  • PART I: Correlation score and probability

  • PART II: Ballistic probability model

  • PART III: How could we implement a probability model in a ballistic system?

  • Conclusion and future work


Part i correlation scores and probability l.jpg

Part I Correlation scores and probability

  • Strengths and limitations of the current correlation score

  • Why are correlation scores hard to interpret?

  • Benefits of a probability “score”


Strength and limitations of the correlation score l.jpg

Strength and limitations of the correlation score

  • In the last 15 years, the correlation score has been in the core of FT’s ballistic systems

  • Strength of a correlation score:

    Useful as a ranking tool

    • Can compare score values computed with the same reference “A” (and same type of mark)

      • Score(A against B) > Score(A against C) means

        B looks more similar to A than C does


Strength and limitations of the correlation score cont d l.jpg

Strength and limitations of the correlation score (Cont’d)

  • Limitations of a correlation score:

    • Correlation score is hard to interpret

    • Not useful as an intrinsic similarity measure

  • Examples:

    • Cannot compare score values computed with different references (same type of mark)

      • Score(A-B) > Score(C-D)

        DOES NOT mean that

        the A-B pair looks more similar than the C-D pair

    • Cannot compare score values computed from different marks

      • Score(A-B) for the Firing Pin > Score(A-B) for BreechFace

        DOES NOT mean

        B looks more similar to A on the FiringPin than on the BreechFace


The score is hard to interpret why l.jpg

The score is hard to interpret. Why?

5 reasons:

  • 1] Different algorithms for different marks

    • Characteristics of the correlatable features and the geometry are very different

      • FP/BF: circular contour and a wide variety of features

      • Ejector/Rimfire: polygonal contour

      • Bullets: stria only

  • 2] Algorithms change over time


The score is hard to interpret why cont d l.jpg

The score is hard to interpret. Why? (Cont’d)

3] No unique cartridge or bullet score

  • More than 1 score per exhibit

    • Cartridge cases :

      • BF/FP/Ejector scores

    • Bullets (Land)

      • MaxPhase2D, PeakPhase2D, PeakScore2D

      • 3DScore

  • Number of score per exhibit expected to increase in the future

    • Cartridge cases: 3D scores

    • Bullets:

      • Added 3D Land score

      • GEA scores?


The score is hard to interpret why cont d9 l.jpg

The score is hard to interpret. Why? (Cont’d)

  • 4] Effect of the database size

    • As the database size increases, the probability to find non matches that look similar to a given reference increases

      • The probability to find a known match in the Top10 decreases even if the score does not change

    • The score value alone is not sufficient. The database size is an important factor as well.

    • “Universal law”, not only in ballistics systems


The score is hard to interpret individual score response l.jpg

The score is hard to interpret. Individual Score Response

  • 5] Each reference has its own “score response”.

    Example:

    • If two cartridges A and B are correlated against the same large database (with no match in it)

      Sometimes get two very different list of scores

      • For example, scores associated with A could be greater then scores associated with B


The score is hard to interpret individual score response cont d l.jpg

The score is hard to interpret. Individual Score Response (Cont’d)

  • Experiment: Correlate 9LG bullets against the same large database (800 non matches) with BulletTRAX-3D

  • Compare their non match score distribution

    • Significant differences

      • high score region

      • position of the peak

  • Each bullet has its own statistical distribution of non match scores

    • No universal “score response” common to all bullets

9LG

Bullet #A

9LG

Bullet #B


Solution convert scores into probabilities l.jpg

Solution: Convert scores into probabilities

  • Each of the previous problems can be solved using probabilities (in principle)

    • Different Algorithms:

      • Probability is a common concept for all score types

    • Algorithms change over time

      • Probability value may still change, but slightly

    • Distinct score response for each bullet/cartridge

      • Probability is a common concept for all exhibits

    • Effect of database size

      • Statistical models based on relevant data could quantify this effect

    • More than 1 score per bullet/cartridge

      • Compute a probability for each score and combine them to find a unique probability for the bullet/cartridge


How could we combine probabilities cartridge case l.jpg

How could we combine probabilities? Cartridge case

  • Assume

    • we have a BF and a FP score for a pair of cartridge cases AND

    • the 2 following probabilities are known

      • P(FP): Confirmed match according to FP

      • P(BF): Confirmed match according to BF

  • 4 possible scenarios

    • Confirmed match according to BOTH FP and BF

    • Confirmed match according to FP ONLY

    • Confirmed match according to BF ONLY

    • Not a confirmed match


How could we combine probabilities cartridge case cont d l.jpg

How could we combine probabilities? Cartridge case (Cont’d)

  • FP/BF marks provide independent information

  • A combined probability is computed by assuming independent information

    • P Combined = 1 – (1-PBF)(1-PFP)

  • Results:

    • A mark with a low probability has no effect on the combined probability

    • As we add marks, the combined probability improves

  • Easy to generalize for 3 independent marks


How could we combine probabilities bullets l.jpg

How could we combine probabilities? Bullets

  • The 4 bullets’ scores are not computed from independent information

    • Are computed from the same areas on the bullet

  • A combined probability cannot be computed by assuming independent information

    • Keep the highest probability only (conservative)


Conclusion part i l.jpg

Conclusion: Part I

  • The probability of being a match is a more meaningful concept than correlation score

  • Using probability solves all problems found with the interpretation of correlation scores

    • Probabilities of individual marks can be combined nicely

  • Challenge: Compute the probability of being a match for individual marks

    • Two main unknowns:

      • How to deal with the individual score response of each cartridge/bullet

      • How to predict the effect of database size


Part ii ballistic probability model l.jpg

Part II Ballistic Probability Model

  • Goal and constraints of the model

  • Hypothesis

  • Tests and results


Statistical model of scores goal constraints l.jpg

Statistical model of scores: Goal & Constraints

  • Project started in 2003

  • Goal: Develop a model which

    • Converts the correlation score of a mark into a probability of being a match

  • Current constraints

    • We only have database of sister pairs

  • Tests with BulletTRAX-3D scores

    • The model should find the same performance as the large database study

    • As the database size increases, the probability to find a known match in the first position should decrease


Ballistic statistical model hypothesis l.jpg

Ballistic Statistical Model: Hypothesis

  • Any mathematical or physical model starts with a small number of hypotheses/laws/axioms

  • Need hypotheses for the (3D bullet) ballistic model

    • Need to find something common to all bullet score distributions

    • However, each bullet has its own score response


Hypothesis cont d l.jpg

Hypothesis (Cont’d)

  • Non Match Statistical distribution

  • Experiment already discussed:

    Correlate 9LG bullets against the same large database (800 non matches)

  • Compare their non match score distribution (3D)

    • Differences

      • in the high score region

      • in the position of the peak

    • Similarity:

      • The distributions have a similar shape

9LG

Bullet #1

9LG

Bullet #2


Hypothesis cont d21 l.jpg

Hypothesis (Cont’d)

  • Core Hypothesis:

    The non match score distribution of all bullets

    • Has the same universal “shape” (up to a shift and stretch factor)

    • This shape is independent of calibre, material and quality of the marks

      Can be broken into two hypotheses

  • Hypothesis I:

    • The non match score distribution of each bullet is fully characterized by only two parameters:

      • its mean (position of the peak)

      • its width

  • Hypothesis II:

    • If we remove the effect of these 2 parameters,

      the non match score distributions of bullets are strictly identical

  • The effect of the 2 parameters is removed as follows

    • Shift the overall distribution at the same peak position for every bullet

    • Shrink or expand the overall distribution to get the same width for every bullet


Hypothesis cont d22 l.jpg

Hypothesis (Cont’d)

  • The effect of the 2 parameters is removed as follow

    • Shift the mean to 0

    • Shrink to unit width

  • Get very similar distributions!

  • Small variations due to limited data

9LG

Bullet #1

9LG

Bullet #2


Ballistic statistical model testing the model l.jpg

Ballistic Statistical Model: Testing the model

  • 4 steps:

    • Compute 3D correlation scores from a large database study with BulletTRAX-3D

      • 4 calibers, 2 materials/compositions

    • Compute the individual parameters for each bullet (Hypothesis I)

      • mean and width of its non match score distribution

    • Determine a Universal Non Match score distribution

      (Hypothesis II)

    • By simulations, predict the performance of the correlation algorithm as a function of database size


Testing the model database general information l.jpg

Testing the model :Database General Information

*Pittsburgh bullets database

(Allegheny County Coroner’s Office

Forensic Laboratory Division)


Testing the model compute individual parameters l.jpg

Testing the model :Compute individual parameters

  • For each bullet

    • get an approximation of the universal distribution (Hypothesis II)

  • The scores are normalized by this process

  • For each bullet:

    • Mean and width are computed

    • The distribution is

      • Shifted the mean to 0

      • Rescaled to unit width


Testing the model define a universal non match distribution l.jpg

Add up the “approximated” universal distributions found for all bullets

Smooth shape even in high score region

Universal Normalized distribution for non match scores

Testing the model :Define a “universal” non match distribution


Testing the model simulations l.jpg

Testing the model :Simulations

  • The simulation reproduces the operations done in a real large database study

  • Real study (with sister pairs)

    • For each reference bullet

      • Introduce its known match in the database of size N

      • Compute all correlation scores between the reference and (N+1) bullets in the database

      • Find the rank of the known match

    • Compute the performance of the correlation algorithm (number of known matches at the first position)


Testing the model simulations cont d l.jpg

Testing the model :Simulations (Cont’d)

  • Simulation:

    • For each reference bullet

      • Select randomly N non match (normalized) correlation scores from the universal score distribution

      • Normalize the (known) score of its known match by using

        • the reference’s individual parameters (mean and width of its non match score distribution)

      • Introduce the normalized score of its known match in the (generated) non-match score list

      • Find the rank of the known match

    • Compute the simulated performance of the correlation algorithm

  • Repeat the same process for several databases sizes N


Testing the model simulations cont d29 l.jpg

Testing the model Simulations (Cont’d)

Probability that the sister is at the first position as a function of its “normalized” score S

  • Dark circles: experimental data

  • Dark curve:

    Result from the model

  • Gray curves: Prediction for other database sizes

8


Testing the model simulations cont d30 l.jpg

Testing the model Simulations (Cont’d)

Summary of the figure

  • If the sister has a “normalized score” = 8

    • The probability to be in first position is

      • 90% for N = 500

      • 70% for N = 2K

      • 20% for N = 10K

  • If we want the sister to be at the first position with a 95% probability,

    • its score must be

      • 9 for N = 500

      • 10 for N = 2K

      • 12 for N = 10K


Part ii summary l.jpg

Part II: Summary

  • A statistical model of non match scores was built

    • a database of 2000 bullets, 4 calibers, 2 compositions/materials

    • 3D correlation on BulletTRAX-3D

  • Hypothesis:

    • The non match score distribution has the same shape for all bullets

      (except for a shift and stretch factor)

  • The model computes the probability that the sister with a given score is in first position

    • The prediction agrees with the actual performance in the large database study

    • Performance decreases as the database size increases


Part iii l.jpg

Part III

How could we implement a probabilistic model in a ballistic system?


How could we implement a probabilistic model in a ballistic system l.jpg

How could we implement a probabilistic model in a ballistic system?

  • Correlate a given bullet against a large database

  • From the (large) list of scores, compute the two characteristic parameters of the reference bullet

    • mean and width of its non match score distribution

  • Compute the probability that the bullet in the first position is a match by using

    • The universal non match score distributions

    • Two characteristic parameters computed previously

    • Actual score of the bullet at the first position

    • Information about match score distributions (unknown yet)


How could we implement a probabilistic model in a ballistic system cont d l.jpg

How could we implement a probabilistic model in a ballistic system? (Cont’d)

  • Repeat the same process for all score types

    • MaxPhase2D, PeakPhase2D, PeakScore2D

    • 3DScore

  • Combine the 4 probabilities into a unique probability for the bullet


Future work l.jpg

Future work

  • Improving the model with new large database studies (new calibers)

  • Test on cartridges

  • Get a better knowledge of sister score distributions

    • The current study was done with sister pairs only

  • Use the model to improve correlation algorithms


  • Login