Multiple instance hidden markov model application to landmine detection in gpr data
Download
1 / 86

Multiple Instance Hidden Markov Model: Application to Landmine Detection in GPR Data - PowerPoint PPT Presentation


  • 115 Views
  • Uploaded on

Multiple Instance Hidden Markov Model: Application to Landmine Detection in GPR Data. Jeremy Bolton, Seniha Yuksel , Paul Gader CSI Laboratory University of Florida. Highlights. Hidden Markov Models (HMMs) are useful tools for landmine detection in GPR imagery

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Multiple Instance Hidden Markov Model: Application to Landmine Detection in GPR Data' - oriana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Multiple instance hidden markov model application to landmine detection in gpr data

Multiple Instance Hidden Markov Model: Application to Landmine Detection in GPR Data

Jeremy Bolton, SenihaYuksel, Paul Gader

CSI Laboratory

University of Florida


Highlights
Highlights Landmine Detection in GPR Data

  • Hidden Markov Models (HMMs) are useful tools for landmine detection in GPR imagery

  • Explicitly incorporating the Multiple Instance Learning (MIL) paradigm in HMM learning is intuitive and effective

  • Classification performance is improved when using the MI-HMM over a standard HMM

  • Results further support the idea that explicitly accounting for the MI scenario may lead to improved learning under class label uncertainty


Outline
Outline Landmine Detection in GPR Data

  • HMMs for Landmine detection in GPR

    • Data

    • Feature Extraction

    • Training

  • MIL Scenario

  • MI-HMM

  • Classification Results


Hmms for landmine detection

HMMs for landmine detection Landmine Detection in GPR Data


Gpr data
GPR Data Landmine Detection in GPR Data

  • GPR data

    • 3d image cube

      • Dt, xt, depth

    • Subsurface objects are observed as hyperbolas


Gpr data feature extraction
GPR Data Feature Extraction Landmine Detection in GPR Data

  • Many features extracted from in GPR data measure the occurrence of an “edge”

    • For the typical HMM algorithm (Gaderet al.),

      • Preprocessing techniques are used to emphasize edges

      • Image morphology and structuring elements can be used to extract edges

Image

Preprocessed

Edge Extraction


4 d edge features
4-d Edge Features Landmine Detection in GPR Data

Edge Extraction


Concept behind the hmm for gpr
Concept behind the HMM for GPR Landmine Detection in GPR Data

  • Using the extracted features (an observation sequence when scanning from left to right in an image) we will attempt to estimate some hidden states


Concept behind the hmm for gpr1
Concept behind the HMM for GPR Landmine Detection in GPR Data


Hmm features
HMM Features Landmine Detection in GPR Data

  • Current AIM viewer by Smock

Image

Feature Image

Rising Edge Feature

Falling Edge Feature


Sampling hmm summary
Sampling HMM Summary Landmine Detection in GPR Data

  • Feature Calculation

    • Dimensions (Not always relevant whether positive or negative diagonal is observed …. Just simply a diagonal is observed)

      • HMMSamp: 2d

    • Down sampling depth

      • HMMSamp: 4

  • HMM Models

    • Number of States

      • HMMSamp : 4

    • Gaussian components per state (Fewer total components for probability calculation)

      • HMMSamp : 1 (recent observation)


Training the hmm
Training the HMM Landmine Detection in GPR Data

  • Xuping Zhang proposed a Gibbs Sampling algorithm for HMM learning

    • But, given an image(s) how do we choose the training sequences?

    • Which sequence(s) do we choose from each image?

  • There is an inherent problem in many image analysis settings due to class label uncertainty per sequence

  • That is, each image has a class label associated with it, but each image has multiple instances of samples or sequences. Which sample(s) is truly indicative of the target?

    • Using standard training techniques this translates to identifying the optimal training set within a set of sequences

    • If an image has N sequences this translates to a search of 2N possibilities


Training sample selection heuristic
Training Sample Selection Heuristic Landmine Detection in GPR Data

  • Currently, an MRF approach (Collins et al.) is used to bound the search to a localized area within the image rather than search all sequences within the image.

    • Reduces search space, but multiple instance problem still exists


Multiple instance learning

Multiple Instance Learning Landmine Detection in GPR Data


Standard learning vs multiple instance learning
Standard Learning vs. Multiple Instance Learning Landmine Detection in GPR Data

  • Standard supervised learning

    • Optimize some model (or learn a target concept) given training samples and corresponding labels

  • MIL

    • Learn a target concept given multiplesets of samples and corresponding labels for the sets.

    • Interpretation: Learning with uncertain labels / noisy teacher


Multiple instance learning mil
Multiple Instance Learning (MIL) Landmine Detection in GPR Data

  • Given:

    • Set of I bags

      • Labeled + or -

  • The ith bag is a set of Ji

    samples in some feature space

  • Interpretation of labels

  • Goal: learn concept

    • What characteristic is common to the positive bags that is not observed in the negative bags


  • Standard learning doesn t always fit gpr example
    Standard Landmine Detection in GPR Datalearning doesn’t always fit: GPR Example

    • Standard Learning

      • Each training sample (feature vector) must have a label

      • But which ones and how many compose the optimal training set?

        • Arduous task: many feature vectors per image and multiple images

        • Difficult to label given GPR echoes, ground truthing errors, etc …

        • Label of each vector may not be known

    EHD: Feature Vector


    Learning from bags
    Learning from Bags Landmine Detection in GPR Data

    • In MIL, a label is attached to a set of samples.

    • A bag is a set of samples

    • A sample within a bag is called an instance.

    • A bag is labeled as positive if and only if at least one of its instances is positive.

    POSITIVE BAGS

    (Each bag is an image)

    NEGATIVE BAGS

    (Each bag is an image)


    Mi learning gpr example

    EHD: Feature Vector Landmine Detection in GPR Data

    MI Learning: GPR Example

    • Multiple Instance Learning

      • Each training bag must have a label

        • No need to label all feature vectors, just identify images (bags) where targets are present

        • Implicitly accounts for class label uncertainty …


    Multiple instance learning hmm mi hmm

    Multiple Instance Learning HMM: MI-HMM Landmine Detection in GPR Data


    Mi hmm
    MI-HMM Landmine Detection in GPR Data

    • In MI-HMM, instances are sequences

    Direction of

    movement

    NEGATIVE BAGS

    POSITIVE BAGS


    Mi hmm1
    MI-HMM Landmine Detection in GPR Data

    • Assuming independence between the bags and assuming the Noisy-OR (Pearl) relationship between the sequences within each bag

    • where


    Mi hmm learning
    MI-HMM learning Landmine Detection in GPR Data

    • Due to the cumbersome nature of the noisy-OR, the parameters of the HMM are learned using Metropolis – Hastings sampling.


    Sampling
    Sampling Landmine Detection in GPR Data

    • HMM parameters are sampled from Dirichlet

    • A new state is accepted or rejected based on the ratio rat iteration t + 1

    • where P is the noisy-or model.


    Discrete observations
    Discrete Observations Landmine Detection in GPR Data

    • Note that since we have chosen a Metropolis Hastings sampling scheme using Dirichlets, our observations must be discretized.


    Mi hmm summary
    MI-HMM Landmine Detection in GPR DataSummary

    • Feature Calculation

      • Dimensions

        • HMMSamp: 2d

        • MI-HMM: 2d features are descretized into 16 symbols

      • Down sampling depth

        • HMMSamp: 4

        • MI-HMM: 4

    • HMM Models

      • Number of States

        • HMMSamp : 4

        • MI-HMM: 4

      • Components per state (Fewer total components for probability calculation)

        • HMMSamp : 1 Gaussian

        • MI-HMM: Discrete mixture over 16 symbols


    Classification results

    Classification Results Landmine Detection in GPR Data


    Mi hmm vs sampling hmm
    MI-HMM Landmine Detection in GPR Datavs Sampling HMM

    • Small Millbrook

    HMM Samp (12,000)

    MI-HMM (100)


    What s the deal with hmm samp
    What’s the deal with HMM Landmine Detection in GPR DataSamp?


    Concluding remarks

    Concluding Remarks Landmine Detection in GPR Data


    Concluding remarks1
    Concluding Remarks Landmine Detection in GPR Data

    • Explicitly incorporating the Multiple Instance Learning (MIL) paradigm in HMM learning is intuitive and effective

    • Classification performance is improved when using the MI-HMM over a standard HMM

      • More effective and efficient

    • Future Work

      • Construct bags without using MRF heuristic

      • Apply to EMI data: spatial uncertainty


    Back up slides

    Back up Slides Landmine Detection in GPR Data


    Standard learning vs multiple instance learning1
    Standard Learning vs. Multiple Instance Learning Landmine Detection in GPR Data

    • Standard supervised learning

      • Optimize some model (or learn a target concept) given training samples and corresponding labels

    • MIL

      • Learn a target concept given multiplesets of samples and corresponding labels for the sets.

      • Interpretation: Learning with uncertain labels / noisy teacher


    Multiple instance learning mil1
    Multiple Instance Learning (MIL) Landmine Detection in GPR Data

    • Given:

      • Set of I bags

        • Labeled + or -

    • The ith bag is a set of Ji

      samples in some feature space

    • Interpretation of labels

  • Goal: learn concept

    • What characteristic is common to the positive bags that is not observed in the negative bags


  • Mil application example gpr

    EHD: Feature Vector Landmine Detection in GPR Data

    MIL Application: Example GPR

    • Collaboration: Frigui, Collins, Torrione

    • Construction of bags

      • Collect 15 EHD feature vectors from the 15 depth bins

      • Mine images = + bags

      • FA images = - bags


    Standard vs mi learning gpr example
    Standard vs. MI Learning: GPR Example Landmine Detection in GPR Data

    • Standard Learning

      • Each training sample (feature vector) must have a label

        • Arduous task

          • many feature vectors per image and multiple images

          • difficult to label given GPR echoes, ground truthing errors, etc …

          • label of each vector may not be known

    EHD: Feature Vector


    Standard vs mi learning gpr example1

    EHD: Feature Vector Landmine Detection in GPR Data

    Standard vs MI Learning: GPR Example

    • Multiple Instance Learning

      • Each training bag must have a label

        • No need to label all feature vectors, just identify images (bags) where targets are present

        • Implicitly accounts for class label uncertainty …



    Random set brief
    Random Set Brief Landmine Detection in GPR Data

    • Random Set


    How can we use random sets for mil

    It is Landmine Detection in GPR DataNOT the case that EACH

    element is NOT the

    target concept

    How can we use Random Sets for MIL?

    • Random set for MIL: Bags are sets

      • Idea of finding commonality of positive bags inherent in random set formulation

        • Sets have an empty intersection or non-empty intersection relationship

        • Find commonality using intersection operator

        • Random sets governing functional is based on intersection operator

          • Capacity functional : T

    A.K.A. : Noisy-OR gate (Pearl 1988)


    Random set functionals
    Random Set Functionals Landmine Detection in GPR Data

    • Capacity functionals for intersection calculation

    • Use germ and grain model to model random set

      • Multiple (J) Concepts

      • Calculate probability of intersection given X and germ and grain pairs:

      • Grains are governed by random radii with assumed cumulative:

    Random Set model parameters

    Germ

    Grain


    Rsf mil germ and grain model

    x Landmine Detection in GPR Data

    T

    x

    T

    T

    x

    x

    x

    x

    T

    T

    x

    x

    x

    RSF-MIL: Germ and Grain Model

    • Positive Bags = blue

    • Negative Bags = orange

    • Distinct shapes = distinct bags



    Multiple concepts disjunction or conjunction
    Multiple Concepts: Landmine Detection in GPR DataDisjunction or Conjunction?

    • Disjunction

      • When you have multiple types of concepts

      • When each instance can indicate the presence of a target

    • Conjunction

      • When you have a target type that is composed of multiple (necessary concepts)

      • When each instance can indicate a concept, but not necessary the composite target type


    Conjunctive rsf mil
    Conjunctive RSF-MIL Landmine Detection in GPR Data

    • Previously Developed Disjunctive RSF-MIL (RSF-MIL-d)

    • Conjunctive RSF-MIL (RSF-MIL-c)

    Noisy-OR combination across concepts and samples

    Standard noisy-OR for one concept j

    Noisy-AND combination across concepts


    Synthetic data experiments
    Synthetic Data Experiments Landmine Detection in GPR Data

    • Extreme Conjunct data set requires that a target bag exhibits two distinct concepts rather than one or none

    AUC (AUC when initialized near solution)


    Application to remote sensing

    Application to Remote Sensing Landmine Detection in GPR Data


    Disjunctive target concepts
    Disjunctive Target Concepts Landmine Detection in GPR Data

    • Using Large overlapping bins (GROSS Extraction) the target concept can be encapsulated within 1 instance: Therefore a disjunctive relationship exists

    Target Concept

    Type 1

    NoisyOR

    Target Concept

    Type 2

    NoisyOR

    OR

    Target Concept

    Type n

    NoisyOR

    Target Concept Present?


    What if we want features with finer granularity
    What if we want features with finer granularity Landmine Detection in GPR Data

    • Fine Extraction

      • More detail about image and more shape information, but may loose disjunctive nature between (multiple) instances

    Constituent Concept 1

    (top of hyperbola)

    NoisyOR

    Target Concept Present?

    AND

    Constituent Concept 2

    (wings of hyperbola)

    NoisyOR

    Our features have more granularity, therefore our concepts may be constituents of a target, rather than encapsulating the target concept


    Gpr experiments
    GPR Experiments Landmine Detection in GPR Data

    • Extensive GPR Data set

      • ~800 targets

      • ~ 5,000 non-targets

    • Experimental Design

      • Run RSF-MIL-d (disjunctive) and RSF-MIL-c (conjunctive)

      • Compare both feature extraction methods

        • Gross extraction: large enough to encompass target concept

        • Fine extraction: Non-overlapping bins

    • Hypothesis

      • RSF-MIL will perform well when using gross extraction whereas RSF-MIL-c will perform well using Fine extraction


    Experimental results
    Experimental Results Landmine Detection in GPR Data

    • Highlights

      • RSF-MIL-d using gross extraction performed best

      • RSF-MIL-c performed better than RSF-MIL-d when using fine extraction

      • Other influencing factors: optimization methods for RSF-MIL-d and RSF-MIL-c are not the same

    Gross Extraction

    Fine Extraction


    Future work
    Future Work Landmine Detection in GPR Data

    • Implement a general form that can learn disjunction or conjunction relationship from the data

    • Implement a general form that can learn the number of concepts

    • Incorporate spatial information

    • Develop an improved optimization scheme for RSF-MIL-C


    Hmm model visualization
    HMM Model Visualization Landmine Detection in GPR Data

    Points =

    Gaussian Component means

    DTXTHMM

    Falling

    Diagonal

    Color =

    State Index

    State index1

    State index 2

    State index 3

    Rising Diagonal

    Transition probabilitiesfrom state to state (red = high probability)

    Initial probabilities

    Pattern Characterized


    Backup slides

    Backup Slides Landmine Detection in GPR Data


    Mil example ahi imagery
    MIL Example (AHI Imagery) Landmine Detection in GPR Data

    • Robust learning tool

      • MIL tools can learn target signature with limited or incomplete ground truth

    Which spectral signature(s) should we use to train a target model or classifier?

    Spectral mixing

    Background signal

    Ground truth not exact


    Mi rvm
    MI-RVM Landmine Detection in GPR Data

    • Addition of set observations and inference using noisy-OR to an RVM model

    • Prior on the weight w


    Svm review
    SVM review Landmine Detection in GPR Data

    • Classifier structure

    • Optimization


    Mi svm discussion
    MI-SVM Discussion Landmine Detection in GPR Data

    • RVM was altered to fit MIL problem by changing the form of the target variable’s posterior to model a noisy-OR gate.

    • SVM can be altered to fit the MIL problem by changing how the margin is calculated

      • Boost the margin between the bag (rather than samples) and decision surface

      • Look for the MI separating linear discriminant

        • There is at least one sample from each bag in the half space


    Mi svm
    mi-SVM Landmine Detection in GPR Data

    • Enforce MI scenario using extra constraints

    At least one sample in each positive bag must have a label of 1.

    Mixed integer program: Must find optimal hyperplane and optimal labeling set

    All samples in each negative bag must have a label of -1.


    Current applications
    Current Applications Landmine Detection in GPR Data

    • Multiple Instance Learning

      • MI Problem

      • MI Applications

    • Multiple Instance Learning: Kernel Machines

      • MI-RVM

      • MI-SVM

    • Current Applications

      • GPR imagery

      • HSI imagery


    Hsi t arget spectra learning
    HSI: Landmine Detection in GPR DataTarget Spectra Learning

    • Given labeled areas of interest: learn target signature

    • Given test areas of interest: classify set of samples


    Overview of mi rvm optimization
    Overview of MI-RVM Optimization Landmine Detection in GPR Data

    • Two step optimization

      • Estimate optimal w, given posterior of w

        • There is no closed form solution for the parameters of the posterior, so a gradient update method is used

        • Iterate until convergence. Then proceed to step 2.

      • Update parameter on prior of w

        • The distribution on the target variable has no specific parameters.

        • Until system convergence, continue at step 1.


    1 optimization of w
    1) Optimization of Landmine Detection in GPR Dataw

    • Optimize posterior (Bayes’ Rule) of w

    • Update weights using Newton-Raphsonmethod


    2 optimization of prior
    2) Optimization of Prior Landmine Detection in GPR Data

    • Optimization of covariance of prior

    • Making a large number of assumptions, diagonal elements of A can be estimated


    Random sets multiple instance learning
    Random Sets: Multiple Instance Learning Landmine Detection in GPR Data

    • Random set framework for multiple instance learning

      • Bags are sets

      • Idea of finding commonality of positive bags inherent in random set formulation

        • Find commonality using intersection operator

        • Random sets governing functional is based on intersection operator


    Mi issues
    MI issues Landmine Detection in GPR Data

    • MIL approaches

      • Some approaches are biased to believe only one sample in each bag caused the target concept

      • Some approaches can only label bags

      • It is not clear whether anything is gained over supervised approaches


    Rsf mil

    x Landmine Detection in GPR Data

    T

    x

    T

    T

    x

    x

    x

    x

    T

    T

    x

    x

    x

    RSF-MIL

    • MIL-like

    • Positive Bags = blue

    • Negative Bags = orange

    • Distinct shapes = distinct bags


    Side note bayesian networks
    Side Note: Bayesian Networks Landmine Detection in GPR Data

    • Noisy-OR Assumption

      • Bayesian Network representation of Noisy-OR

      • Polytree: singly connected DAG


    Side note
    Side Note Landmine Detection in GPR Data

    • Full Bayesian network may be intractable

      • Occurrence of causal factors are rare (sparse co-occurrence)

        • So assume polytree

        • So assume result has boolean relationship with causal factors

      • Absorb I, X and A into one node, governed by randomness of I

        • These assumptions greatly simplify inference calculation

        • Calculate Z based on probabilities rather than constructing a distribution using X


    Diverse density dd
    Diverse Density (DD) Landmine Detection in GPR Data

    • Probabilistic Approach

      • Goal:

        • Standard statistics approaches identify areas in a feature space with high density of target samples and low density of non-target samples

        • DD: identify areas in a feature space with a high “density” of samples from EACH of the postitive bags (“diverse”), and low density of samples from negative bags.

          • Identify attributes or characteristics similar to positive bags, dissimilar with negative bags

      • Assume t is a target characterization

      • Goal:

      • Assuming the bags are conditionally independent


    Diverse density

    It is Landmine Detection in GPR DataNOT the case that EACH

    element is NOT the

    target concept

    Diverse Density

    • Calculation (Noisy-OR Model):

    • Optimization


    Random set brief1
    Random Set Brief Landmine Detection in GPR Data

    • Random Set


    Random set functionals1
    Random Set Functionals Landmine Detection in GPR Data

    • Capacity and avoidance functionals

      • Given a germ and grain model

      • Assumed random radii


    When disjunction makes sense
    When disjunction makes sense Landmine Detection in GPR Data

    • Using Large overlapping bins the target concept can be encapsulated within 1 instance: Therefore a disjunctive relationship exists

    Target Concept Present

    OR


    Theoretical and developmental progress
    Theoretical and Developmental Progress Landmine Detection in GPR Data

    • Previous Optimization:

      • Did not necessarily promote

        diverse density

  • Current optimization

    • Better for context learning and MIL

  • Previously no feature relevance or selection (hypersphere)

    • Improvement: included learned weights on each feature dimension

    • Previous TO DO list

    • Improve Existing Code

      • Develop joint optimization for context learning and MIL

    • Apply MIL approaches (broad scale)

      • Learn similarities between feature sets of mines

      • Aid in training existing algos: find “best” EHD features for training / testing

      • Construct set-based classifiers?


    How do we impose the mi scenario diverse density maron et al

    It is Landmine Detection in GPR DataNOT the case that EACH

    element is NOT the

    target concept

    How do we impose the MI scenario?: Diverse Density (Maronet al.)

    • Calculation (Noisy-OR Model):

      • Inherent in Random Set formulation

    • Optimization

      • Combo of exhaustive search and gradient ascent


    How can we use random sets for mil1
    How can we use Random Sets for MIL? Landmine Detection in GPR Data

    • Random set for MIL: Bags are sets

      • Idea of finding commonality of positive bags inherent in random set formulation

        • Sets have an empty intersection or non-empty intersection relationship

        • Find commonality using intersection operator

        • Random sets governing functional is based on intersection operator

    • Example:

    Bags with target

    {l,a,e,i,o,p,u,f}

    {f,b,a,e,i,z,o,u}

    {a,b,c,i,o,u,e,p,f}

    {a,f,t,e,i,u,o,d,v}

    Bags without target

    {s,r,n,m,p,l}

    {z,s,w,t,g,n,c}

    {f,p,k,r}

    {q,x,z,c,v}

    {p,l,f}

    intersection

    union

    Target concept =

    \

    {a,e,i,o,u,f}

    {f,s,r,n,m,p,l,z,w,g,n,c,v,q,k}

    = {a,e,i,o,u}


    ad