This presentation is the property of its rightful owner.
1 / 18

Segmentation PowerPoint PPT Presentation

Segmentation. Divide the image into segments. Each segment: Looks uniform Belongs to a single object. Have some uniform attributes. All the pixel related to it are connected. …. Main approaches. Histogram-based segmentation Region-based segmentation Edge detection Region growing

Segmentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Segmentation

Divide the image into segments.

Each segment:

Looks uniform

Belongs to a single object.

Have some uniform attributes.

All the pixel related to it are connected.

Main approaches

Histogram-based segmentation

Region-based segmentation

Edge detection

Region growing

Region splitting and merging.

Clustering

K-means

Mean shift

Motion segmentation

Text & Background

Problem: we are given an image of a paper, and we would like to extract the text from the image.

Thresholding: define a threshold T such that each pixel x,y where I(x,y)<T is “text”.

How do we determine the threshold ?

Just choose 128 as a threshold (problematic for dark images)

Use the median/mean (both are not good, as most of the paper is white)

Histogram-based threshold

Compute the gray level histogram of the image.

Find two “clusters”: black and white.

Minimizing the L2 error:

Select initial estimate T

Segment the image using T.

Compute the average gray level of each segment mb,mw

Compute a new threshold value: T = ½ (mb+mw)

Continue until convergence.

We are already familiar with this algorithm !

Problems with this approach

Noise

Many holes and discontinuities in the segmentation.

Changes in the illumination

we do not use spatial information.

Some of the problems can be solved using image processing techniques. For example, we can enhance the result using morphological operations.

Yet – How can we overcome the changes in the illumination ?

Divide the image into sub-images.

Assume that the illumination in each sub-images is constant.

Use a different threshold for each sub-image.

Alternatively – use a running window (and use the threshold of the window only for the central pixel )

Problems:

Rapid illumination changes.

Regions without text: we can try to recognize that these regions is unimodal.

Region-based segmentation

We would like to use spatial information.

We assume that neighboring pixels tend to belong to the same segment (not always true)

Edge detection: looking for the boundaries of the segments.

Problem: edges usually do not determine close contours. We can try to do it with edge linking (as in Canny’s edge detector)

Region-based segmentation

Basic Formulation

Let R represent the entire image region.

Segmentation: Partitioning R into n subgroups Ri s.t:

a)

b) is a connected region

c)

d)

e)

P is the partition predicate

Region growing

Choose a group of points as initial regions.

Expand the regions to neighboring pixels using a heuristic:

Color distance from the neighbors.

The total error in the region (till a certain threshold):

Variance

Sum of the differences between neighbors.

Maximal difference from a central pixel.

In some cases, we can also use structural information: the region size and shape.

In this way we can handle regions with a smoothly varying gray level or color.

Question: How do we choose the starting points ? It is less important if we also can merge regions.

Region merging and splitting

In region merging we start with small regions (it can be pixels), and iteratively merge regions which are similar.

In region splitting, we start with the whole image, and split regions which are not uniform.

These methods can be combined. Formally:

Choose a predicate P.

Split into disjoint regions any region Ri for which

Merge any adjacent regions Ri and Rj for which

Stop when no further merging and splitting is possible.

R

R21

R22

R1

R23

R1

R2

R3

R4

R24

R3

R4

R21

R23

R24

R22

• With quadtree, one can use a variation of the split & merge scheme:

• Only at the final stage: merge regions.

Segmentation as clustering

• Address the image as a set of points in the n-dimensional space:

• Gray level images: p=(x,y,I(x,y)) in R3

• Color images: p =(x,y,R(x,y),G(x,y),B(x,y)) in R5

• Texture: p= (x,y,vector_of_fetures)

• Color Histograms: p=(R(x,y),G(x,y),B(x,y)) in R3.we ignore the spatial information.

• From this stage, we forget the meaning of each coordinate. We deal with arbitrary set of points.

• Therefore, we first need to “normalize” the features (for example - convert a color image to the appropriate linear space representation)

Again, we can use splitting & merging

Here, we merge each time the closest neighbors.

K-means

• Idea:

• Determine the number of clusters

• •Find the cluster centers and point-cluster correspondences to minimize error

• Problem: Exhaustive search is too expensive.

• Solution: We will use instead an iterative search.[Recall the ideal quantization procedure.]

Algorithm

– fix cluster centers; allocate points to closest cluster

– fix allocation; compute best cluster centers

Error function =

Mean Shift

• K-means is a powerful and popular method for clustering. However:

• It assumes a pre-determined number of clusters

• It “likes” compact clusters. Sometimes, we are looking for long but continues clusters.

• Mean Shift:

• Determine a window size (usually small).

• For each point p:

• Compute a weighted mean of the shift in the window:

• Set p := p + m

• Continue until convergence.

• At the end, use a more standard clustering method.

Mean Shift (cont’)

• This method is based on the assumption that points are more and more dense as we are getting near the cluster “central mass”.

Motion segmentation

• Background subtraction:

• Assumes the existence of a dominant background.

• Optical flow (use the motion vectors as features)

• Multi model motion:

• Divide the image to layers such that in each layer, there exist a parametric motion model.