MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision

1 / 209

# MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision - PowerPoint PPT Presentation

MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision. The science of form and structure the science of form, that of the outer form, inner structure, and development of living organisms and their parts about changing/counting regions/shapes

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' MATHEMATICS OF BINARY MORPHOLOGY and APPLICATIONS IN Vision' - ross-curry

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

MATHEMATICS OF BINARY MORPHOLOGY

and

APPLICATIONS IN Vision

The science of form and structure

the science of form, that of the outer form, inner structure, and development of living organisms and their parts

Among other applications it is used to pre- or post-process images

via filtering, thinning and pruning

Count regions (granules)

number of black regions

Estimate size of regions

area calculations

In general; what is “Morphology”?
• Smooth region edges
• create line drawing of face
• Force shapes onto region edges
• curve into a square
What is Morphology in computer vision ?
• Morphology generally concerned with shape and properties of objects.
• Used for segmentation and feature extraction.
• Segmentation = used for cleaning binary objects.
• Two basic operations
• erosion (opening)
• dilation (closing)
Morphological operations and algebras
• Different definitions in the textbooks
• Different implementations in the image processing programs.
• The original definition, based on set theory, is made by J. Serra in 1982.
• Defined for binary images - binary operations (boolean, set-theoretical)
• Can be used on grayscale images - multiple-valued logic operations
Morphological operations on a PC
• Various but slightly different implementations in
• Scion
• Paint Shop Pro
• Corel Photopaint
• mm

Try them, it is a lot of fun

### Mathematical Morphology - Set-theoretic representation for binary shapes

Binary Morphology
• Morphological operators are used to prepare binary (thresholded) images for object segmentation/recognition
• Binary images often suffer from noise (specifically salt-and-pepper noise)
• Binary regions also suffer from noise (isolated black pixels in a white region). Can also have cracks, picket fence occlusions, etc.
• Dilation and erosion are two binary morphological operations that can assist with these problems.
Goals of morphological operations:
• 1. Simplifies image data
• 2. Preserves essential shape characteristics
• 3. Eliminates noise
• 4. Permits the underlying shape to be identified and optimally reconstructed from their distorted, noisy forms
What is the mathematical morphology ?
• 1. An approach for processing digital image based on its shape
• 2. A mathematical tool for investigating geometric structure in image
• The language of morphology is set theory.

Mathematical morphology is extension to set theory.

Importance of Shape in Processing and Analysis
• Shape is a prime carrier of information in machine vision
• For instance, the following directly correlate with shape:
• identification of objects
• object features
• assembly defects

Set Union (overlapping objects):

Shape Operators
• Shapes are usually combined by means of :
• Set Difference based on Set intersection (occluded objects):

Set difference

Set intersection

Morphological Operations based on combining base operations
• The primary morphological operations are dilationanderosion
• More complicated morphological operators can be designed by means of combining erosions and dilations

We will use combinations of union, complement, intersection, erosion, dilation, translation...

Let us illustrate them and explain how to combine

Libraries of Structuring Elements
• Application specific structuring elements created by the user

Notation

x

-2 -1 0 1 2

-2 -1 0 1 2

B

y

A special set :

the structuring element

Origin at center in this case, but not necessarily centered nor symmetric

X

No necessarily compact

nor filled

3*3 structuring element, see next slide how it works

Explanation of Dilation

Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X.

X

difference

dilation

B

Notation for Dilation

Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X.

How to formulate this definition ?

1) Literal translation

Mathematical definition of dilation

2) Better : from Minkowski’s sum of sets

Another Mathematical definition of dilation uses the concept of Minkowski’s sum

B is ingeneral not the same as B

The Concept

of Minkowski Sum

Minkowski’s Sum

Definition of Minkowski’s sum of sets S and B :

l

Minkowski’s Sum

l

Another View at Dilation

Dilation :

l

Dilation

Dilation

Comparison of Dilation and Minkowski sum

Dilation :

Bx =

x and b are points

Minkowski sum

Dilation and Minkowski Set

Dilation and Minkowski Set are denoted by + or by 

No unified notation

Dilation is not the Minkowski’s sum

l

b

b

b

b

Dilation

l

Dilation

l

B is not the same as B

Dilation vs SE
• Erosion shrinks
• Dilation expands binary regions
• Can be used to fill in gaps or cracks in binary images

structuring Element ( SE )

• If the point at the origin of the structuring element is set in the underlying image, then all the points that are set in the SE are also set in the image
• Basically, its like OR’ing the SE into the image
Dilation fills holes
• Fills in holes.
• Smoothes object boundaries.
• Adds an extra outer ring of pixels onto object boundary, ie, object becomes slightly larger.
Possible problems with Morphological Operators
• Erosion and dilation clean image but leave objects either smaller or larger than their original size.
• Opening and closing perform same functions as erosion and dilation but object size remains the same.
More Erode and Dilate Examples

Input Image

Dilated

Eroded

Dilation explained pixed by pixel

Denotes origin of B i.e. its (0,0)

Denotes origin of A i.e. its (0,0)

B

A

Dilation explained by shape of A

Shape of A repeated without shift

B

Shape of A repeated with shift

A

1. fills in valleys between spiky regions

2. increases geometrical area of object

3. sets background pixels adjacent to object\'s contour to object\'s value

4. smoothes small negative grey level regions

Properties of Dilation

objects are light (white in binary)

Dilation does the following:

Structuring Element for Dilation

Single point in Image replaced with this in the Result

Definition of Dilation: Mathematically
• Dilation is the operation that combines two sets using vector addition of set elements.
• Let A and B are subsets in 2-D space. A: image undergoing analysis, B: Structuring element, denotes dilation
Dilation versus translation
• Let A be a Subset of and .
• The translation of A by x is defined as:
• The dilation of A by B can be computed as the union of translation of A by the elements of B

x is a vector

Dilation versus translation, illustrated

Element (0,0)

Shift vector (0,1)

Shift vector (0,0)

B

Dilation using Union Formula

Center of the circle

This circle will create one point

This circle will create no point

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

Mathematical Properties of Dilation
• Commutative
• Associative
• Extensivity
• Dilation is increasing

Illustrated in next slide

Illustration of Extensitivity of Dilation

A

B

Replaced with

Here 0 does not belong to B and A is not included in A B

More Properties of Dilation
• Translation Invariance
• Linearity
• Containment
• Decomposition of structuring element
Dilation
• The dilation operator takes two pieces of data as input
• A binary image, which is to be dilated
• A structuring element (or kernel), which determines the behavior of the morphological operation
• Suppose that X is the set of Euclidean coordinates of the input image, and K is the set of coordinates of the structuring element
• Let Kx denote the translation of K so that its origin is at x.
• The DILATION of X by K is simply the set of all points x such that the intersection of Kx with X is non-empty

1

1

1

1

1

1

1

1

1

Dilation

Example: Suppose that the structuring element is a 3x3 square with the origin at its center

{ (-1,-1), (0,-1), (1,-1),

(-1,0), (0,0), (1,0),

( 1,1), (0,1), (1,1) }

K =

X =

Dilation

Example: Suppose that the structuring element is a 3x3 square with the origin at its center

Note: Foreground pixels are represented by a color and background pixels are empty

Dilation

Structuring element

Input

output

Dilation

output

Dilation

output

Example of Erosion

Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.

difference

erosion

Notation for Erosion

Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.

How to formulate this definition ?

1) Literal translation

Erosion

2) Better : from Minkowski’s substraction of sets

Minkowski’s substraction

Notation for Erosion

BINARY MORPHOLOGY

Minkowski’s substraction of sets

Erosion

Erosion with other structuring elements

Did not belong to X

When the new SE is included in old SE then a larger area is created

Common structuring elements shapes

= origin

x

y

disk

circle

segments 1 pixel wide

Note that here :

points

Problem in BINARY MORPHOLOGY

using Minkowski Sum

First we calculate the operation in parentheses to obtain a diamond

PROBLEM BINARY MORPHOLOGY

next we calculate the external operation to obtain a hexagon

ANOTHER EXAMPLE OF EROSION

Where d is a diameter

Problem :

<d/2

d/2

d

Implementation of dilation:

very low computational cost

0

1 (or >0)

Logical or

Implementation of erosion:

very low computational cost

0

1

Logical and

Erosion
• (Minkowski subtraction)
• The contraction of a binary region (aka, region shrinking)
• Use a structuring element on binary image data to produce a new binary image
• Structuring elements (SE) are simply patterns that are matched in the image
• It is useful to explain operation of erosion and dilation in different ways.
Typical Uses of Erosion
• Removes isolated noisy pixels.
• Smoothes object boundary.
• Removes the outer layer of object pixels, ie, object becomes slightly smaller.
Erosion removes spiky edges

objects are light (white in binary)

decreases geometrical area of object

sets contour pixels of object to background value

smoothes small positive grey level regions

Properties of Erosion:
Erosion explained pixel by pixel

A

B

How It Works?
• During erosion, a pixel is turned on at the image pixel under the structuring element origin only when the pixels of the structuring element match the pixels in the image
• Both ON and OFF pixels should match.
• This example erodes regions horizontally from the right.
Mathematical Definition of Erosion
• Erosion is the morphological dual to dilation.
• It combines two sets using the vector subtraction of set elements.
• Let denotes the erosion of A by B
Erosion in terms of other operations:
• Erosion can also be defined in terms of translation
• In terms of intersection

Observe that vector here is negative

Reminder - this was A

Observe negative translation

Because of negative shift the origin is here

Erosion formula and intuitive example

Center of B is here and adds a point

Center here will not add a point to the Result

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

Properties of Erosion
• Erosion is not commutative!
• Extensivity
• Erosion is dereasing
• Chain rule

The chain rule is as sharp operator in Cube Calculus used in logic synthesis. There are more similarities of these algebras

Properties of Erosion
• Translation Invariance
• Linearity
• Containment
• Decomposition of structuring element
Duality Relationship between erosion and dilation
• Dilation and Erosion transformation bear a marked similarity, in that what one does to image foreground and the other does for the image background.
• , the reflection of B, is defined as
• Erosion and Dilation Duality Theorem

Observe negative value which is mirror image reflection of B

Similar but not identical to De Morgan rule in Boolean Algebra

Erosion as Dual of Dilation
• Erosion is the dual of dilation
• i.e. eroding foreground pixels is equivalent to dilating the background pixels.
Easily visualized on binary image

Template created with known origin

Template stepped over entire image

similar to correlation

Dilation

if origin == 1 -> template unioned

resultant image is large than original

Erosion

only if whole template matches image

origin = 1, result is smaller than original

Duality Relationship between erosion and dilation

Another look at duality

One more view at Erosion with examples
• To compute the erosion of a binary input image by the structuring element
• For each foreground pixel superimpose the structuring element
• If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is
• Otherwise, if any of the corresponding pixels in the image are background, however, the input pixel is set to background value
Erosion example with dilation and negation

We want to calculate this

We dilate with negation

Erosion

.. And we negate the result

We obtain the same thing as from definition

Morphological Operations in terms of more general neighborhoods

This exists in Matlab

Erode and Dilate in terms of more general neighborhoods

Yet another loook at Duality Relationship between erosion and dilation

Erode and Binary Contour in Matlab

Erosion can be used to find contour

Dilation can be also used for it - think how?

Edge detection

This subtraction is set theoretical

Dilate - original

Now you need to invert the image

There are more methods for edge detection

Opening & Closing
• Opening and Closing are two important operators from mathematical morphology
• They are both derived from the fundamental operations of erosion and dilation
• They are normally applied to binary images

Close = Dilate next Erode

Open = Erode next Dilate

Open and Close

Original image

eroded

dilated

dilated

eroded

Open

Close

OPENING

Opening :

also

OPENING

difference

• Supresses :
• small islands
• ithsmus (narrow unions)
• narrow caps
Open
• An erosion followed by a dilation
• It serves to eliminate noise
• Does not significantly change an object’s size
Comparison of Opening and Erosion
• Opening is defined as an erosion followed by a dilation using the same structuring element
• The basic effect of an opening is similar to erosion
• Tends to remove some of the foreground pixels from the edges of regions of foreground pixels
• Less destructive than erosion
• The exact operation is determined by a structuring element.
Opening Example
• What combination of erosion and dilation gives:
• cleaned binary image
• object is the same size as in original

Original

Opening Example Cont
• Erode original image.
• Dilate eroded image.
• Smooths object boundaries, eliminates noise (isolated pixels) and maintains object size.

Original

Erode

Dilate

One more example of Opening
• Erosion can be used to eliminate small clumps of undesirable foreground pixels, e.g. “salt noise”
• However, it affects all regions of foreground pixels indiscriminately
• Opening gets around this by performing both an erosion and a dilation on the image

EXAMPLE OF CLOSING

Closing :

also

• Supresses :
• small lakes (holes)
• channels (narrow separations)
• narrow bays

BINARY MORPHOLOGY

Closing previous image with other structuring elements

With bigger rectangle like this

With smaller cross like this

original

negated

threshold

closing

opening

and withthreshold

Application :

segmentation of microstructures (Matlab Help)

PROPERTIES IN BINARY MORPHOLOGY

• Properties
• all of them are increasing :
• opening and closing are idempotent :

EXTENSIVE VERSUS ANTI-EXTENSIVE OPERATIONS

• dilation and closing are extensive operations
• erosion and opening are anti-extensive operations:

DUALITIES OF MORPHOLOGICAL OPERATORS

• dualityof erosion-dilation, opening-closing,...

Decomposition of structuring elements

operations with big structuring elements can be done

by a succession of operations with small s.e’s

HIT-OR-MISS

Hit-or-miss :

Bi-phase structuring element

“Hit” part

(white)

“Miss” part

(black)

HIT or MISS FOR ISOLATED POINTS

Looks for pixel configurations :

background

foreground

doesn’t matter

ISOLATED POINTS

isolated points at4 connectivity

Close
• Dilation followed by erosion
• Serves to close up cracks in objects and holes due to pepper noise
• Does not significantly change object size
More examples of Closing
• What combination of erosion and dilation gives:
• cleaned binary image
• object is the same size as in original

Original

More examples of Closing cont
• Dilate original image.
• Erode dilated image.
• Smooths object boundaries, eliminates noise (holes) and maintains object size.

Dilate

Erode

Original

Closing as dual to Opening
• Closing, like its dual operator opening, is derived from the fundamental operations of erosion and dilation.
• Normally applied to binary images
• Tends to enlarge the boundaries of foreground regions
• Less destructive of the original boundary shape
• The exact operation is determined by a structuring element.
Closing is opening in revers
• Closing is opening performed in reverse.
• It is defined simply as a dilation followed by an erosion using the same
Mathematical Definitions of Opening and Closing
• Opening and closing are iteratively applied dilation and erosion

Opening

Closing

Relation of Opening and Closing

Difference is only in corners

Opening and Closing are idempotent
• Their reapplication has not further effects to the previously transformed result
Properties of Opening and Closing
• Translation invariance
• Antiextensivity of opening
• Extensivity of closing
• Duality

Structuring

Element

Pablo Picasso, Pass with the Cape, 1960

Structuring

Element

Example of Closing

Thinning

and Thickening

Thinning and Thickening

Thinning :

Thickenning :

• Depending on the structuring elements (actually, series
• of them), very different results can be achieved :
• Prunning
• Skeletons
• Zone of influence
• Convex hull
• ...

Prunning at 4 connectivity

Prunning at 4 connectivity : remove end points by a sequence of thinnings

This point is removed with dark green neighbors

1 iteration =

IDEMPOTENCE shown as a result of thinning

1st iteration

2nd iteration

3rd iteration: idempotence

Other thinning operations

background

doesn’t matter

foreground

USING EROSION TO FIND CONTOURS

Contours of binary regions :

erosion

difference

CONTOURS with different connectivity patterns

8-connectivity

contour

4-connectivity

contour

4-connectivity

8-connectivity

Important for perimeter computation.

Use of thickening: Convex hull

ii. Convex hull : union of thickenings, each up to idempotence

Original shaper

Union of four thickenings

iii. Skeleton :

Maximal disk : disk centered at x, Dx, such that

DxX and no other Dy contains it .

Skeleton : union of centers of maximal disks.

PROBLEMS with skeletons

• Problems :
• Instability : infinitessimal variations in the border of X
• cause large deviations of the skeleton
• not necessarily connex even though X connex
• good approximations provided by thinning with
• special series of structuring elements

Example of iterative thinning with 8 masks

result of

1st iteration

2nd iteration reaches

idempotence

Thinning

with thickening

20 iterations of thinning color white

40 iterations

Thickening

color white

Some sort of region clustering

Skeletonization for OCR

BINARY MORPHOLOGY

Application : skeletonization for OCR by graph matching

skeletonization

vectorization

Hit-or-Miss

and 3 rotations

skeletonization

Application : skeletonization for OCR by graph matching

Calculation of Geodesic zones of influence (GZI)

X set of n connex components {Xi},i=1..n.

The zone of influence of Xi , Z(Xi) ,is the set of points closer to some point of Xi than to a point of any other component.

Also, Voronoi partition.

Dual to skeleton.

PROGRAMMING

OF MORPHOLOGICAL OPERATIONS

Morphological Filtering
• Main idea of Morphological Filtering:
• Examine the geometrical structure of an image by matching it with small patterns called structuring elements at various locations
• By varying the size and shapeof the matching patterns, we can extract useful informationabout the shape of the different parts of the imageand their interrelations.
• Combine set-theoretical and morphological operations:
Example 1: Morphological filtering
• Noisy image will break down OCR systems

Noisy image

Clean original image

Rank Filter Median

Input

1 operation

2 operations

Postprocessing
• Opening followed by closing.
• Removes noise and smoothes boundaries.
Postprocessing
• Opening followed by closing.
• Removes noise and smoothes boundaries.

erosion

dilation

Removal of Border Objects

Marker is the border itself

Summary on Morphological Approaches
• Mathematical morphology is an approach for processing digital image based on its shape
• The language of morphology is set theory
• The basic morphological operations are erosion and dilation
• Morphological filtering can be developed to extract useful shape information
• Methods can be extended to more values and more dimensions
• Nice mathematics can be formulated - non-linear
Conclusion
• Segmentation separates an image into regions.
• Use of histograms for brightness based segmentation.
• Peak corresponds to object.
• Height of peak corresponds to size of object.
• If global image histogram is multimodal, local image region histogram may be bimodal.
• Local thresholds can give better segmentation.
Conclusion
• Postprocessing uses morphological operators.
• Same as convolution only use Boolean operators instead of multiply and add.
• Erosion clears noise, makes smaller.
• Dilation fills in holes, makes larger.
• Postprocessing
• Opening and closing to clean binary images.
• Repeated erosion with special rule produces skeleton.
Problems 1 - 6
• 1. Write LISP or C++ program for dilation of binary images
• 2. Modify it to do erosions (few types)
• 3. Modify it to perform shift and exor operation and shift and min operation
• 4. Generalize to multi-valued algebra
• 5. Create a comprehensive theory of multi-valued morphological algebra and its algorithms (publishable).
• 6. Write a program for inspection of Printed Circuit Boards using morphological algebra.
Problem 7.
• Electric Outlet Extraction has been done using a combination of Canny Edge Detection and Hough Transforms
• Write a LISP program that will use only basic morphological methods for this application.
Image Processing for electric outlet, how?

Alpha filtering

DPC compression

Perimeter

Fractal

Gaussian Filter

Band Pass Filter

Homomorphic Filtering

Contrast

Sharper

Least Square Restoration

Warping

Dilation

• Currently there are many, many ways to approach this problem
• Segmentation
• Edge Detection
• DPC compression
• FFT
• IFFT
• DFT
• Thinning
• Growing
• Haar Transform
• Hex Rotate
Image Processing, how?
• Create morphological equivalents of other image processing methods.
• New, publishable, use outlet problem as example to illustrate
Problem 8. Openings and Closings as examples.
• The solution here is to follow up one operation with the other.
• An opening is defined as an erosion operation followed by dilation using the same structuring element.
• Similarly, a closing is dilation followed by erosion.
• Define and implement other combined operations.
Problems 9 - 12.
• 9. Generalize binary morphological algebra from 2 dimensional to 3 dimensional images. What are the applications.
• 10. Write software for 9.
• 11. Generalize your generalized multi-valued morphological algebra to 3 or more dimensions, theoretically, find properties and theorems like those from this lecture.
• 12. Write software for 11.
Problem 13
• Mathematical morphology uses the concept of structuring elements to analyze image features.
• A structuring element is a set of pixels in some arrangement that can extract shape information from an image.
• Typical structuring elements include rectangles, lines, and circles.
• Think about other structuring elements and their applications.
Morphological Operations: Matlab

BWMORPH Perform morphological operations on binary image.

BW2 = BWMORPH(BW1,OPERATION) applies a specific morphological operation to the binary image BW1.

BW2 = BWMORPH(BW1,OPERATION,N) applies the operation N times. N can be Inf, in which case the operation is repeated until the image no longer changes.

OPERATION is a string that can have one of these values:

\'close\' Perform binary closure (dilation followed by erosion)

\'dilate\' Perform dilation using the structuring elementones(3)

\'erode\' Perform erosion using the structuring elementones(3)

\'fill\' Fill isolated interior pixels (0\'s surrounded by1\'s)

\'open\' Perform binary opening (erosion followed bydilation)

\'skel\' With N = Inf, remove pixels on the boundariesof objects without allowing objects to break apart

• demos/demo9morph/
Sources

D.A. Forsyth, University of New Mexico,

Qigong Zheng, Language and Media Processing Lab

Center for Automation Research

University of Maryland College Park

October 31, 2000

John Miller

Matt Roach

J. W. V. Miller and K. D. Whitehead

The University of Michigan-Dearborn

Spencer Lustor

Light Works Inc. C. Rössl, L. Kobbelt, H.-P. Seidel, Max-Planck Institute for, Computer Science, Saarbrücken, Germany

LBA-PC4

Howard Schultz

Shreekanth Mandayam

ECE Department

Rowan University

D.A. Forsyth, University of New Mexico

More recent Sources

• Howie Choset
• G.D. Hager,
• Z. Dodds,
• Dinesh Mocha