1 / 209

# MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision - PowerPoint PPT Presentation

MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision. The science of form and structure the science of form, that of the outer form, inner structure, and development of living organisms and their parts about changing/counting regions/shapes

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision' - ross-curry

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

and

APPLICATIONS IN Vision

the science of form, that of the outer form, inner structure, and development of living organisms and their parts

Among other applications it is used to pre- or post-process images

via filtering, thinning and pruning

Count regions (granules)

number of black regions

Estimate size of regions

area calculations

In general; what is “Morphology”?

• Smooth region edges

• create line drawing of face

• Force shapes onto region edges

• curve into a square

• Morphology generally concerned with shape and properties of objects.

• Used for segmentation and feature extraction.

• Segmentation = used for cleaning binary objects.

• Two basic operations

• erosion (opening)

• dilation (closing)

• Different definitions in the textbooks

• Different implementations in the image processing programs.

• The original definition, based on set theory, is made by J. Serra in 1982.

• Defined for binary images - binary operations (boolean, set-theoretical)

• Can be used on grayscale images - multiple-valued logic operations

• Various but slightly different implementations in

• Scion

• Paint Shop Pro

• Corel Photopaint

• mm

Try them, it is a lot of fun

### Mathematical Morphology - Set-theoretic representation for binary shapes

• Morphological operators are used to prepare binary (thresholded) images for object segmentation/recognition

• Binary images often suffer from noise (specifically salt-and-pepper noise)

• Binary regions also suffer from noise (isolated black pixels in a white region). Can also have cracks, picket fence occlusions, etc.

• Dilation and erosion are two binary morphological operations that can assist with these problems.

• 1. Simplifies image data

• 2. Preserves essential shape characteristics

• 3. Eliminates noise

• 4. Permits the underlying shape to be identified and optimally reconstructed from their distorted, noisy forms

• 1. An approach for processing digital image based on its shape

• 2. A mathematical tool for investigating geometric structure in image

• The language of morphology is set theory.

Mathematical morphology is extension to set theory.

Importance of Shape in Processing and Analysis

• Shape is a prime carrier of information in machine vision

• For instance, the following directly correlate with shape:

• identification of objects

• object features

• assembly defects

Shape Operators

• Shapes are usually combined by means of :

• Set Difference based on Set intersection (occluded objects):

Set difference

Set intersection

• The primary morphological operations are dilationanderosion

• More complicated morphological operators can be designed by means of combining erosions and dilations

We will use combinations of union, complement, intersection, erosion, dilation, translation...

Let us illustrate them and explain how to combine

• Application specific structuring elements created by the user

x

-2 -1 0 1 2

-2 -1 0 1 2

B

y

A special set :

the structuring element

Origin at center in this case, but not necessarily centered nor symmetric

X

No necessarily compact

nor filled

3*3 structuring element, see next slide how it works

Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X.

X

difference

dilation

B

Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X.

How to formulate this definition ?

1) Literal translation

Mathematical definition of dilation

2) Better : from Minkowski’s sum of sets

Another Mathematical definition of dilation uses the concept of Minkowski’s sum

B is ingeneral not the same as B

of Minkowski Sum

Definition of Minkowski’s sum of sets S and B :

l

Minkowski’s Sum

l

Dilation :

l

Dilation

Dilation

Dilation :

Bx =

x and b are points

Minkowski sum

Dilation and only to top and to rightMinkowski Set

Dilation and Minkowski Set are denoted by + or by 

No unified notation

Dilation is only to top and to rightnot the Minkowski’s sum

Minkowski’s Sum

l

Dilation is only to top and to rightnot the Minkowski’s sum

l

b

b

b

b

Dilation

l

Dilation

l

B is not the same as B

Dilation vs only to top and to rightSE

• Erosion shrinks

• Dilation expands binary regions

• Can be used to fill in gaps or cracks in binary images

structuring Element ( SE )

• If the point at the origin of the structuring element is set in the underlying image, then all the points that are set in the SE are also set in the image

• Basically, its like OR’ing the SE into the image

Dilation fills holes only to top and to right

• Fills in holes.

• Smoothes object boundaries.

• Adds an extra outer ring of pixels onto object boundary, ie, object becomes slightly larger.

Main Applications of Dilation only to top and to right

Dilation example only to top and to right

Possible problems with Morphological Operators only to top and to right

• Erosion and dilation clean image but leave objects either smaller or larger than their original size.

• Opening and closing perform same functions as erosion and dilation but object size remains the same.

More Erode and Dilate Examples only to top and to right

Input Image

Dilated

Eroded

Dilation explained pixed by pixel only to top and to right

Denotes origin of B i.e. its (0,0)

Denotes origin of A i.e. its (0,0)

B

A

Dilation explained by shape of A only to top and to right

Shape of A repeated without shift

B

Shape of A repeated with shift

A

1. fills in valleys between spiky regions only to top and to right

2. increases geometrical area of object

3. sets background pixels adjacent to object's contour to object's value

4. smoothes small negative grey level regions

Properties of Dilation

objects are light (white in binary)

Dilation does the following:

Structuring Element for Dilation only to top and to right

Length 6

Length 5

Structuring Element for Dilation only to top and to right

Structuring Element for Dilation only to top and to right

Single point in Image replaced with this in the Result

Structuring Element for Dilation only to top and to right

Definition of Dilation: Mathematically only to top and to right

• Dilation is the operation that combines two sets using vector addition of set elements.

• Let A and B are subsets in 2-D space. A: image undergoing analysis, B: Structuring element, denotes dilation

Dilation versus translation only to top and to right

• Let A be a Subset of and .

• The translation of A by x is defined as:

• The dilation of A by B can be computed as the union of translation of A by the elements of B

x is a vector

Dilation versus translation, illustrated only to top and to right

Element (0,0)

Shift vector (0,1)

Shift vector (0,0)

B

Dilation using Union Formula only to top and to right

Center of the circle

This circle will create one point

This circle will create no point

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

• Commutative

• Associative

• Extensivity

• Dilation is increasing

Illustrated in next slide

A

B

Replaced with

Here 0 does not belong to B and A is not included in A B

More Properties of Dilation elements

• Translation Invariance

• Linearity

• Containment

• Decomposition of structuring element

Dilation elements

• The dilation operator takes two pieces of data as input

• A binary image, which is to be dilated

• A structuring element (or kernel), which determines the behavior of the morphological operation

• Suppose that X is the set of Euclidean coordinates of the input image, and K is the set of coordinates of the structuring element

• Let Kx denote the translation of K so that its origin is at x.

• The DILATION of X by K is simply the set of all points x such that the intersection of Kx with X is non-empty

1 elements

1

1

1

1

1

1

1

1

Dilation

Example: Suppose that the structuring element is a 3x3 square with the origin at its center

{ (-1,-1), (0,-1), (1,-1),

(-1,0), (0,0), (1,0),

( 1,1), (0,1), (1,1) }

K =

X =

Dilation elements

Example: Suppose that the structuring element is a 3x3 square with the origin at its center

Note: Foreground pixels are represented by a color and background pixels are empty

Dilation elements

Structuring element

Input

output

Dilation elements

output

Dilation elements

output

Dilation elements

Dilation elements

Erosion elements

Example of Erosion elements

Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.

difference

erosion

Notation for Erosion elements

Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.

How to formulate this definition ?

1) Literal translation

Erosion

2) Better : from Minkowski’s substraction of sets

Minkowski’s substraction

Notation for Erosion elements

BINARY MORPHOLOGY

Minkowski’s substraction of sets

Erosion

Did not belong to X

When the new SE is included in old SE then a larger area is created

Common structuring elements elementsshapes

= origin

x

y

disk

circle

segments 1 pixel wide

Note that here :

points

Problem in BINARY MORPHOLOGY elements

using Minkowski Sum

First we calculate the operation in parentheses to obtain a diamond

PROBLEM BINARY MORPHOLOGY elements

next we calculate the external operation to obtain a hexagon

ANOTHER EXAMPLE OF EROSION elements

Where d is a diameter

Problem :

<d/2

d/2

d

Implementation of dilation: elements

very low computational cost

0

1 (or >0)

Logical or

Implementation of elementserosion:

very low computational cost

0

1

Logical and

More on Erosion elements

Erosion elements

• (Minkowski subtraction)

• The contraction of a binary region (aka, region shrinking)

• Use a structuring element on binary image data to produce a new binary image

• Structuring elements (SE) are simply patterns that are matched in the image

• It is useful to explain operation of erosion and dilation in different ways.

Typical Uses of Erosion elements

• Removes isolated noisy pixels.

• Smoothes object boundary.

• Removes the outer layer of object pixels, ie, object becomes slightly smaller.

Erosion removes elementsspiky edges

objects are light (white in binary)

decreases geometrical area of object

sets contour pixels of object to background value

smoothes small positive grey level regions

Properties of Erosion:

Erosion Example elements

Erosion explained pixel by pixel elements

A

B

How It Works? elements

• During erosion, a pixel is turned on at the image pixel under the structuring element origin only when the pixels of the structuring element match the pixels in the image

• Both ON and OFF pixels should match.

• This example erodes regions horizontally from the right.

• Erosion is the morphological dual to dilation.

• It combines two sets using the vector subtraction of set elements.

• Let denotes the erosion of A by B

• Erosion can also be defined in terms of translation

• In terms of intersection

Observe that vector here is negative

elements

Reminder - this was A

Observe negative translation

Because of negative shift the origin is here

Erosion formula and intuitive example translation

Center of B is here and adds a point

Center here will not add a point to the Result

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

Properties of Erosion elements

• Erosion is not commutative!

• Extensivity

• Erosion is dereasing

• Chain rule

The chain rule is as sharp operator in Cube Calculus used in logic synthesis. There are more similarities of these algebras

Properties of Erosion elements

• Translation Invariance

• Linearity

• Containment

• Decomposition of structuring element

• Dilation and Erosion transformation bear a marked similarity, in that what one does to image foreground and the other does for the image background.

• , the reflection of B, is defined as

• Erosion and Dilation Duality Theorem

Observe negative value which is mirror image reflection of B

Similar but not identical to De Morgan rule in Boolean Algebra

Erosion as Dual of Dilation elements

• Erosion is the dual of dilation

• i.e. eroding foreground pixels is equivalent to dilating the background pixels.

Template created with known origin

Template stepped over entire image

similar to correlation

Dilation

if origin == 1 -> template unioned

resultant image is large than original

Erosion

only if whole template matches image

origin = 1, result is smaller than original

Duality Relationship between erosion and dilation

Another look at duality

• To compute the erosion of a binary input image by the structuring element

• For each foreground pixel superimpose the structuring element

• If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is

• Otherwise, if any of the corresponding pixels in the image are background, however, the input pixel is set to background value

Erosion elements

We want to calculate this

We dilate with negation

Erosion elements

.. And we negate the result

We obtain the same thing as from definition

Morphological Operations in terms of more general neighborhoods

This exists in Matlab

Yet another loook at Duality Relationship between erosion and dilation

Boundary Extraction neighborhoods

Erode and Binary Contour in Matlab neighborhoods

Erosion can be used to find contour

Dilation can be also used for it - think how?

Edge detection neighborhoods

This subtraction is set theoretical

Dilate - original

Now you need to invert the image

There are more methods for edge detection

Opening & Closing neighborhoods

• Opening and Closing are two important operators from mathematical morphology

• They are both derived from the fundamental operations of erosion and dilation

• They are normally applied to binary images

Close = Dilate neighborhoodsnext Erode

Open = Erode next Dilate

Open and Close

Original image

eroded

dilated

dilated

eroded

Open

Close

OPENING neighborhoods

OPENING neighborhoods

Opening :

also

OPENING

difference

• Supresses :

• small islands

• ithsmus (narrow unions)

• narrow caps

Open neighborhoods

• An erosion followed by a dilation

• It serves to eliminate noise

• Does not significantly change an object’s size

Comparison of Opening and Erosion neighborhoods

• Opening is defined as an erosion followed by a dilation using the same structuring element

• The basic effect of an opening is similar to erosion

• Tends to remove some of the foreground pixels from the edges of regions of foreground pixels

• Less destructive than erosion

• The exact operation is determined by a structuring element.

Opening Example neighborhoods

• What combination of erosion and dilation gives:

• cleaned binary image

• object is the same size as in original

Original

Opening Example Cont neighborhoods

• Erode original image.

• Dilate eroded image.

• Smooths object boundaries, eliminates noise (isolated pixels) and maintains object size.

Original

Erode

Dilate

One more example of Opening neighborhoods

• Erosion can be used to eliminate small clumps of undesirable foreground pixels, e.g. “salt noise”

• However, it affects all regions of foreground pixels indiscriminately

• Opening gets around this by performing both an erosion and a dilation on the image

CLOSING neighborhoods

EXAMPLE OF CLOSING neighborhoods

Closing :

also

• Supresses :

• small lakes (holes)

• channels (narrow separations)

• narrow bays

BINARY MORPHOLOGY neighborhoods

Closing previous image with other structuring elements

With bigger rectangle like this

With smaller cross like this

Application : shape smoothing and noise filtering neighborhoods

Papilary lines recognition

original

negated

threshold

closing

opening

and withthreshold

Application :

segmentation of microstructures (Matlab Help)

PROPERTIES IN BINARY MORPHOLOGY neighborhoods

• Properties

• all of them are increasing :

• opening and closing are idempotent :

EXTENSIVE VERSUS ANTI-EXTENSIVE OPERATIONS neighborhoods

• dilation and closing are extensive operations

• erosion and opening are anti-extensive operations:

DUALITIES OF MORPHOLOGICAL OPERATORS neighborhoods

• dualityof erosion-dilation, opening-closing,...

operations with big structuring elements can be done

by a succession of operations with small s.e’s

HIT-OR-MISS neighborhoods

Hit-or-miss :

Bi-phase structuring element

“Hit” part

(white)

“Miss” part

(black)

HIT or MISS FOR ISOLATED POINTS neighborhoods

Looks for pixel configurations :

background

foreground

doesn’t matter

ISOLATED POINTS neighborhoods

isolated points at4 connectivity

Close neighborhoods

• Dilation followed by erosion

• Serves to close up cracks in objects and holes due to pepper noise

• Does not significantly change object size

More examples of Closing neighborhoods

• What combination of erosion and dilation gives:

• cleaned binary image

• object is the same size as in original

Original

More examples of Closing cont neighborhoods

• Dilate original image.

• Erode dilated image.

• Smooths object boundaries, eliminates noise (holes) and maintains object size.

Dilate

Erode

Original

Closing as dual to Opening neighborhoods

• Closing, like its dual operator opening, is derived from the fundamental operations of erosion and dilation.

• Normally applied to binary images

• Tends to enlarge the boundaries of foreground regions

• Less destructive of the original boundary shape

• The exact operation is determined by a structuring element.

Closing is opening in revers neighborhoods

• Closing is opening performed in reverse.

• It is defined simply as a dilation followed by an erosion using the same

Mathematical Definitions of Opening and Closing neighborhoods

• Opening and closing are iteratively applied dilation and erosion

Opening

Closing

Relation of Opening and Closing neighborhoods

Difference is only in corners

Opening and Closing are neighborhoodsidempotent

• Their reapplication has not further effects to the previously transformed result

Properties of Opening and Closing neighborhoods

• Translation invariance

• Antiextensivity of opening

• Extensivity of closing

• Duality

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

Structuring

Element

Example of Closing

Thinning elements

and Thickening

Thinning and Thickening elements

Thinning :

Thickenning :

• Depending on the structuring elements (actually, series

• of them), very different results can be achieved :

• Prunning

• Skeletons

• Zone of influence

• Convex hull

• ...

Prunning at 4 connectivity elements

Prunning at 4 connectivity : remove end points by a sequence of thinnings

This point is removed with dark green neighbors

1 iteration =

1st iteration

2nd iteration

3rd iteration: idempotence

Other thinning operations elements

background

doesn’t matter

foreground

USING EROSION TO FIND CONTOURS elements

Contours of binary regions :

erosion

difference

8-connectivity

contour

4-connectivity

contour

4-connectivity

8-connectivity

Important for perimeter computation.

Use of thickening: elementsConvex hull

ii. Convex hull : union of thickenings, each up to idempotence

Original shaper

Union of four thickenings

iii. Skeleton : elements

Maximal disk : disk centered at x, Dx, such that

DxX and no other Dy contains it .

Skeleton : union of centers of maximal disks.

PROBLEMS with skeletons elements

• Problems :

• Instability : infinitessimal variations in the border of X

• cause large deviations of the skeleton

• not necessarily connex even though X connex

• good approximations provided by thinning with

• special series of structuring elements

1st

iteration

result of

1st iteration

2nd iteration reaches

idempotence

Thinning elements

with thickening

20 iterations of thinning color white

40 iterations

Thickening

color white

Some sort of region clustering

Skeletonization for OCR elements

BINARY MORPHOLOGY

Application : skeletonization for OCR by graph matching

skeletonization

vectorization

Hit-or-Miss elements

and 3 rotations

skeletonization

Application : skeletonization for OCR by graph matching

X set of n connex components {Xi},i=1..n.

The zone of influence of Xi , Z(Xi) ,is the set of points closer to some point of Xi than to a point of any other component.

Also, Voronoi partition.

Dual to skeleton.

thr

erosion

7x7

GZI

and

opening

5x5

thr

erosion

7x7

dist

thresholding

thinning

smoothing

opening

PROGRAMMING elements

OF MORPHOLOGICAL OPERATIONS

USING LISP elements

USING LISP elements

USING LISP elements

USING LISP elements

USING LISP elements

Morphological Filtering elements

• Main idea of Morphological Filtering:

• Examine the geometrical structure of an image by matching it with small patterns called structuring elements at various locations

• By varying the size and shapeof the matching patterns, we can extract useful informationabout the shape of the different parts of the imageand their interrelations.

• Combine set-theoretical and morphological operations:

Example 1: elements Morphological filtering

• Noisy image will break down OCR systems

Noisy image

Clean original image

Morphological filtering (MF)

Restored image

Rank Filter Median elements

Input

1 operation

2 operations

Postprocessing elements

• Opening followed by closing.

• Removes noise and smoothes boundaries.

Postprocessing elements

• Opening followed by closing.

• Removes noise and smoothes boundaries.

erosion elements

dilation

Removal of Border Objects elements

Marker is the border itself

• Mathematical morphology is an approach for processing digital image based on its shape

• The language of morphology is set theory

• The basic morphological operations are erosion and dilation

• Morphological filtering can be developed to extract useful shape information

• Methods can be extended to more values and more dimensions

• Nice mathematics can be formulated - non-linear

Conclusion elements

• Segmentation separates an image into regions.

• Use of histograms for brightness based segmentation.

• Peak corresponds to object.

• Height of peak corresponds to size of object.

• If global image histogram is multimodal, local image region histogram may be bimodal.

• Local thresholds can give better segmentation.

Conclusion elements

• Postprocessing uses morphological operators.

• Same as convolution only use Boolean operators instead of multiply and add.

• Erosion clears noise, makes smaller.

• Dilation fills in holes, makes larger.

• Postprocessing

• Opening and closing to clean binary images.

• Repeated erosion with special rule produces skeleton.

Problems 1 - 6 elements

• 1. Write LISP or C++ program for dilation of binary images

• 2. Modify it to do erosions (few types)

• 3. Modify it to perform shift and exor operation and shift and min operation

• 4. Generalize to multi-valued algebra

• 5. Create a comprehensive theory of multi-valued morphological algebra and its algorithms (publishable).

• 6. Write a program for inspection of Printed Circuit Boards using morphological algebra.

Problem 7. elements

• Electric Outlet Extraction has been done using a combination of Canny Edge Detection and Hough Transforms

• Write a LISP program that will use only basic morphological methods for this application.

Alpha filtering

DPC compression

Perimeter

Fractal

Gaussian Filter

Band Pass Filter

Homomorphic Filtering

Contrast

Sharper

Least Square Restoration

Warping

Dilation

• Currently there are many, many ways to approach this problem

• Segmentation

• Edge Detection

• DPC compression

• FFT

• IFFT

• DFT

• Thinning

• Growing

• Haar Transform

• Hex Rotate

Image Processing, how? elements

• Create morphological equivalents of other image processing methods.

• New, publishable, use outlet problem as example to illustrate

• The solution here is to follow up one operation with the other.

• An opening is defined as an erosion operation followed by dilation using the same structuring element.

• Similarly, a closing is dilation followed by erosion.

• Define and implement other combined operations.

Problems 9 - 12. elements

• 9. Generalize binary morphological algebra from 2 dimensional to 3 dimensional images. What are the applications.

• 10. Write software for 9.

• 11. Generalize your generalized multi-valued morphological algebra to 3 or more dimensions, theoretically, find properties and theorems like those from this lecture.

• 12. Write software for 11.

Problem 13 elements

• Mathematical morphology uses the concept of structuring elements to analyze image features.

• A structuring element is a set of pixels in some arrangement that can extract shape information from an image.

• Typical structuring elements include rectangles, lines, and circles.

• Think about other structuring elements and their applications.

Morphological Operations: Matlab elements

BWMORPH Perform morphological operations on binary image.

BW2 = BWMORPH(BW1,OPERATION) applies a specific morphological operation to the binary image BW1.

BW2 = BWMORPH(BW1,OPERATION,N) applies the operation N times. N can be Inf, in which case the operation is repeated until the image no longer changes.

OPERATION is a string that can have one of these values:

'close' Perform binary closure (dilation followed by erosion)

'dilate' Perform dilation using the structuring elementones(3)

'erode' Perform erosion using the structuring elementones(3)

'fill' Fill isolated interior pixels (0's surrounded by1's)

'open' Perform binary opening (erosion followed bydilation)

'skel' With N = Inf, remove pixels on the boundariesof objects without allowing objects to break apart

• demos/demo9morph/

Sources elements

D.A. Forsyth, University of New Mexico,

Qigong Zheng, Language and Media Processing Lab

Center for Automation Research

University of Maryland College Park

October 31, 2000

John Miller

Matt Roach

J. W. V. Miller and K. D. Whitehead

The University of Michigan-Dearborn

Spencer Lustor

Light Works Inc. C. Rössl, L. Kobbelt, H.-P. Seidel, Max-Planck Institute for, Computer Science, Saarbrücken, Germany

LBA-PC4

Howard Schultz

Shreekanth Mandayam

ECE Department

Rowan University

D.A. Forsyth, University of New Mexico

More recent Sources elements

• Howie Choset

• G.D. Hager,

• Z. Dodds,

• Dinesh Mocha