Pde methods for image segmentation and shape analysis
Download
1 / 64

PDE methods for Image Segmentation and Shape Analysis: - PowerPoint PPT Presentation


  • 771 Views
  • Uploaded on

PDE methods for Image Segmentation and Shape Analysis: From the Brain to the Prostate and Back presented by John Melonakos – NAMIC Core 1 Workshop – 30/May/2007 Outline Bhattacharyya Segmentation Segmentation Results --------------------------------------- Shape Analysis

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PDE methods for Image Segmentation and Shape Analysis:' - ostinmannual


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Pde methods for image segmentation and shape analysis l.jpg

PDE methods for Image Segmentation and Shape Analysis:

From the Brain to the Prostate and Backpresented by John Melonakos – NAMIC Core 1 Workshop – 30/May/2007


Outline l.jpg
Outline

  • Bhattacharyya Segmentation

  • Segmentation Results

    ---------------------------------------

  • Shape Analysis

  • Shape-Driven Segmentation


Contributors l.jpg
Contributors

  • Georgia Tech-

    • Yogesh Rathi, Sam Dambreville, Oleg Michailovich, Jimi Malcolm, Allen Tannenbaum


Publications l.jpg
Publications

  • S. Dambreville, Y. Rathi, and A. Tannenbaum. A framework for Image Segmentation using Shape Models and Kernel Space Shape Priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, (to appear 2007).

  • O. Michailovich, Y. Rathi, and A. Tannenbaum. Image Segmentation using Active Contours Driven by Informaion-Based Criteria. IEEE Transactions on Image Processing, (to appear 2007).

  • Y. Rathi, O. Michailovich, and A. Tannenbaum. Segmenting images on the Tensor Manifold. In CVPR, 2007.

  • Eric Pichon, Allen Tannenbaum, and Ron Kikinis. A statistically based surface evolution method for medical image segmentation: presentation and validation. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), volume 2, pages 711-720, 2003. Note: Best student presentation in image segmentation award.

  • Y. Rathi, O. Michailovich, J. Malcolm, and A. Tannenbaum. Seeing the Unseen: Segmenting with Distributions. In Intl. Conf. Signal and Image Processing, 2006.

  • J. Malcolm, Y. Rathi, A. Tannenbaum. Graph cut segmentation with nonlinear shape priors. In Intl. Conf. Signal and Image Processing, 2007.


Segmentation hierarchy l.jpg

Threshold-based

Edge-based

Region-based

Parametric methods

Implicit methods

Segmentation Hierarchy

Parameterized representation of the curve (shape)

Implicit representation of the curve using level sets


Geometric active contours l.jpg
Geometric Active Contours

Image

Use Calculus of variations


Our contributions l.jpg
Our Contributions

  • Segmentation by separating intensity based probability distributions (not just intensity moments as in previous works).

  • Novel formulation of the Bhattacharyya distance in the level set framework so as to optimally separate the region inside and outside the evolving contour.


Bhattacharyya distance l.jpg

The Bhattacharyya distance gives a measure of similarity between two distributions:

Bhattacharyya Distance

where z  Z is any photometric variable like intensity, color vector or tensors.

  • B can also be thought of as the cosine of the angle between two vectors.


Bhattacharyya distance9 l.jpg
Bhattacharyya Distance between two distributions:

Let x  R2 specify the co-ordinates in the image plane and I :  R2 Z be a mapping from image plane to the space of photometric variable Z. Then the pdf is given by:

This is the nonparametric density estimate of the pdf of z given the kernel K.


In the level set framework l.jpg
In the Level Set Framework between two distributions:

The pdf’s written in terms of the level set function  is given by :


The first variation l.jpg
The First Variation between two distributions:

For segmentation purposes, we would like to minimize the Bhattacharyya distance. This is achieved using calculus of variations, by taking the first variation of B as follows :


The first variation cont l.jpg
The First Variation (cont.) between two distributions:

The first variation of Pin and Pout is given by :

where,


Resulting pde l.jpg

Plugging in all the components, we get the following PDE (partial differential equation) for separating the distributions :

Resulting PDE


Additional terms l.jpg

In numerical experiments, an additional regularizing term is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Additional Terms


Outline15 l.jpg
Outline is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Bhattacharyya Segmentation

  • Segmentation Results

    ---------------------------------------

  • Shape Analysis

  • Shape-Driven Segmentation


Segmenting the caudate nucleus l.jpg
Segmenting the is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:Caudate Nucleus


Caudate nucleus l.jpg
Caudate Nucleus is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:


Zebra l.jpg
Zebra is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:


The unseen l.jpg
The Unseen! is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:


The unseen20 l.jpg
The Unseen! is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Template Image

Generated Image

Toy example: Region inside and outside was obtained by sampling from a Rayleigh distribution with the same mean and variance.


The unseen21 l.jpg
The Unseen! is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Starting distribution

Final distribution

Actual distribution


Application to tensors l.jpg
Application to Tensors is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Intensity is not enough to segment several types of images.

  • Diffusion Tensor MRI images have become common, where at each pixel a tensor is computed from a set of gradients.

Color coded Fractional Anisotropy image


Structure tensors l.jpg
Structure Tensors is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Structure tensor reveals many features like edges, corners or texture of an image.

  • A structure tensor for a scalar valued image I is given by: (K is a Gaussian kernel)

Color structure tensor is given by:


The tensor manifold l.jpg
The Tensor Manifold is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • The space of n x n positive definite symmetric matrices, is not a vector space, but forms a manifold (a cone).

  • Many past methods by Wang-Vemuri, Lenglet et.al., have however assumed the tensor space to be Euclidean. The active contour based segmentation was performed under this assumption.

Structure tensor space for a typical image.


Riemannian vs euclidean manifold l.jpg
Riemannian vs Euclidean Manifold is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Euclidean distance d(A,B) = d(A,C) = d(C,B) = d1

Riemannian distance dr(A,B) = d(A,C) + d(C,B) = 2d1

Thus, under Euclidean manifold assumption, one obtains an erroneous estimate for mean and variance used in many active contour based segmentation algorithms.


Basic riemannian geometry l.jpg
Basic Riemannian Geometry is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

M

Y

Log Map

x

TxM

Y’

For a tensor manifold (cone), TxM is the set

of all symmetric matrices and forms a vector space.


Tensor space l.jpg
Tensor Space is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • A recent method proposed by Lenglet et.al. (2006), incorporates the Riemannian geometry of the tensor space and performs segmentation by assuming a Gaussian distribution of the object and background.

  • By using the Bhattacharyya distance and taking into account the Riemannian structure of the tensor manifold, we propose to extend the above segmentation technique to any arbitrary and non-analytic probability distribution.


Segmentation framework l.jpg
Segmentation Framework is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Compute Mean on the Riemannian Manifold

Map all points onto the Tangent Space TM at the mean

Perform Curve Evolution using the PDE described earlier.

Compute Target points or bins

Euclidean Space

Accepted for publication in IEEE CVPR 2007


Slide29 l.jpg
Duck is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Segmentation using Bhattacharyya flow, but using Riemannian metric

Segmentation using Bhattacharyya flow, but assuming Euclidean distance between tensors


Tiger l.jpg
Tiger is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Segmentation using Bhattacharyya flow, but using Riemannian metric

Segmentation using Bhattacharyya flow, but assuming Euclidean distance between tensors


Butterfly l.jpg
Butterfly is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Color structure tensor:

Segmentation assuming Riemannian metric

Segmentation assuming Euclidean metric


Segmentation summary l.jpg
Segmentation Summary is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • No assumption on the distribution of the object or background.

  • Computationally very fast, since we only need to update the probability distribution instead of having to map each point in the image from Riemannian space to tangent space after each iteration to compute the mean and variance (under a Gaussian assumption).


Outline33 l.jpg
Outline is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Bhattacharyya Segmentation

  • Segmentation Results

    ---------------------------------------

  • Shape Analysis

  • Shape-Driven Segmentation


Contributors34 l.jpg
Contributors is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Georgia Tech-

    • Delphine Nain, Xavier LeFaucheur, Yi Gao, Allen Tannenbaum


Publications35 l.jpg
Publications is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • D. Nain, S. Haker, A. Tannenbaum. Multiscale 3D shape representation and segmentation using spherical wavelets. IEEE Trans. Medical Imaging, 26 (2007). pp 598-618.

  • D. Nain, S. Haker, A. Bobick, and A. Tannenbaum. Shape-Driven 3D Segmentation using using Spherical Wavelets. In Proceedings of MICCAI, Copenhagen, 2006. Note: Best Student Paper Award in the category Segmentation and Registration.

  • D. Nain, S. Haker, A. Bobick, and A. Tannenbaum. Multiscale 3D Shape Analysis using Spherical Wavelets. In Proceedings of MICCAI, Palm Springs, 2005.


Overview l.jpg
Overview is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • 3D, Parametric, Data-driven prior

Caudate nucleus dataset

Prostate dataset


Overview of asm l.jpg
Overview of ASM is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • K shapes in training set, N landmarks


Limitations of asm l.jpg
Limitations of ASM is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Rank of the covariance matrix DDTis at most K-1

  • Small training set: only first K-1 major variations captured by shape prior

  • E.g. Reconstruction, given new shape s

Ground Truth

Reconstructed with ASM shape prior


Multi scale prior l.jpg
Multi-scale prior is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Hierarchical decomposition: shape is represented at different scales [Davatzikos03]

  • Learn variations at each scale


The algorithm l.jpg
The Algorithm is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Step 1: Find Landmarks

    • [Conformal Mapping]

  • Step 2: Multi-scale representation

    • [Spherical Wavelets]

  • Step 3: Find independent bands of variation

    • [Spectral Graph Partitioning]


1 shape registration and re meshing l.jpg
[1] Shape Registration and Re-meshing is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Nonlinear area-preserving mapping [Brechbuhler95], Conformal mapping [Haker04]


2 spherical wavelets l.jpg
[2] Spherical Wavelets is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

A function decomposed in wavelet

space is uniquely described by a

Weighted sum of scaling functions

and wavelet functions that are

localized in space and scale

Wavelet, level 1

Wavelet, level 2

Wavelet, level 3

Scaling, level 0

Spherical scaling and wavelet functions are defined on a multi-resolution grid


2 spherical wavelets43 l.jpg
[2] Spherical Wavelets is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

is signal on the sphere

Analysis:

Synthesis:

In matrix notation:


2 shape representation l.jpg

j=0 is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

j=1

j=2

j=3

[2] Shape Representation

After the registration step, all shapes have the required mesh structure

Given a shape Sk, we find a 3 1D signals:

We take the wavelet transform of

each signal and represent the shape as:

Shape representation using a weighted combination of the lowest resolution scaling functions and wavelet functions up to jth resolution

Original Shape


2 compression l.jpg
[2] Compression is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • At each scale, we can truncate least significant coefficients based on spectrum analysis of population

  • Results in local compression at each scale

Compression: from 2562 to 649 coefficients, mean error 5.10-3


3 scale space prior l.jpg
[3] Scale-Space Prior is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Previous approach [Davatzikos 03]

We propose a more principled approach where

for each scale, we cluster highly correlated coefficients

into a band, with the constraint that coefficients

across bands have minimum cross-correlation


Band decomposition l.jpg
Band Decomposition is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

1 2 3 4 5 6

2 3 6 1 4 5

1

2

3

4

5

6

2

3

6

1

4

5

Covariance Matrix

level 1

Rearranged Covariance Matrix

level 1


Band decomposition48 l.jpg
Band Decomposition is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Spectral Graph Partitioning technique [Shi00]

  • Fully connected graph G = (V,E) where nodes V are wavelet coefficients for a particular scale

  • Weight on each edge: w(i, j) is covariance between coefficients i and j

  • Stopping criterion: validating whether the subdivided band correspond to two independent distributions based on KL divergence


Band decomposition49 l.jpg
Band Decomposition is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Color is influence of coefficients

in a band: red (high), blue (none)


Building the prior l.jpg
Building the Prior is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Assuming K shapes in training set, for each band, we obtain (K-1) eigenvectors

  • In total we have B(K-1) eigenvectors, where B is number of bands


Experiments l.jpg
Experiments is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Dataset of N samples randomly into T training samples and [N − T] testing samples, where T = [5, 10, 25]

  • Reconstruction task:

  • Test:

    • Compare to ASM, other wavelet band decomposition

    • Effect of noise

    • Effect of truncation


Results l.jpg
Results is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

GT

Noise

Noise

GT

WAV rec. from GT

WAV rec. from Noise

WAV rec. from GT

WAV rec. from Noise

ASM rec. from GT

ASM rec. from Noise

ASM rec. from GT

ASM rec. from Noise


Results53 l.jpg
Results is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Reconstruction from Ground Truth

Reconstruction from Noisy Shape


Outline54 l.jpg
Outline is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Bhattacharyya Segmentation

  • Segmentation Results

    ---------------------------------------

  • Shape Analysis

  • Shape-Driven Segmentation


Shape driven segmentation l.jpg
Shape-Driven Segmentation is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

End goal is to derive a parametric surface evolution equation by evolving the wavelet coefficients directly so that we can include the shape prior directly in the flow


Segmentation via shape prior l.jpg
Segmentation via Shape Prior is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

Shape Representation

  • Probability Distribution

  • Learn shape space (evectors U)

  • Learn bounds within shape space

Pose: Rotations,

Translations, Scaling

shape

Segmentation

Likelihood

Evolve , p

Shape Prior

Constrain , p

Segmented Shape


Energy minimization l.jpg
Energy Minimization is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Region-based energy

Region inside evolving

surface

Data term

[Rousson05]

Points on

Evolving surface

Points on

Evolving surface


Evolution l.jpg
Evolution is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Update equations

step size

Run until

  • Start with lowest resolutions

  • Run for 1 iteration

  • Constrain 

  • Add next resolution s when

  • Coarse to fine evolution


Experiments59 l.jpg
Experiments is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Evolution with PDM shape prior (Active Shape Model)

  • Evolution with WDM shape prior

  • Two types image input:

    • Using Ground Truth image data (binary): to test convergence

    • Using real image data

  • Quantitative measurements:

    • Compare to Ground truth (manually segmented)

  • Details:

    • caudate nucleus shapes from MRI scans

    • training set of 24 shapes, testing set of 5 shapes

    • 4 subdivision levels, 16 bands in the shape prior

    • Start with mean shape, mean position


Ground truth example l.jpg
Ground Truth Example is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Image data is binary

GT in red; ASM in blue

GT in red; Mscale in blue


Caudate nucleus example l.jpg
Caudate Nucleus Example is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

GT in red; ASM in yellow

GT in red; Mscale in blue


Caudate nucleus example62 l.jpg
Caudate Nucleus Example is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • MRI image intensity

GT in red; ASM in blue

GT in red; Mscale in blue


Conclusions speculations l.jpg
Conclusions (Speculations) is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:

  • Geodesic tractography (tomorrow)

  • Fast non-rigid registration (tomorrow)

  • Estimation and filtering techniques from tracking?


Questions l.jpg
Questions? is added to the resulting PDE that penalizes the length of the contour making it smooth. Thus, the final PDE is given by:


ad