1 / 18

Segmentation - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Segmentation. Divide the image into segments. Each segment: Looks uniform Belongs to a single object. Have some uniform attributes. All the pixel related to it are connected. …. Main approaches. Histogram-based segmentation Region-based segmentation Edge detection Region growing

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentationdownload


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript



Divide the image into segments.

Each segment:

Looks uniform

Belongs to a single object.

Have some uniform attributes.

All the pixel related to it are connected.

Main approaches

Main approaches

Histogram-based segmentation

Region-based segmentation

Edge detection

Region growing

Region splitting and merging.



Mean shift

Motion segmentation

Text background

Text & Background

Problem: we are given an image of a paper, and we would like to extract the text from the image.

Thresholding: define a threshold T such that each pixel x,y where I(x,y)<T is “text”.

How do we determine the threshold ?

Just choose 128 as a threshold (problematic for dark images)

Use the median/mean (both are not good, as most of the paper is white)

Histogram based threshold

Histogram-based threshold

Compute the gray level histogram of the image.

Find two “clusters”: black and white.

Minimizing the L2 error:

Select initial estimate T

Segment the image using T.

Compute the average gray level of each segment mb,mw

Compute a new threshold value: T = ½ (mb+mw)

Continue until convergence.

We are already familiar with this algorithm !

Problems with this approach

Problems with this approach


Many holes and discontinuities in the segmentation.

Changes in the illumination

we do not use spatial information.

Some of the problems can be solved using image processing techniques. For example, we can enhance the result using morphological operations.

Yet – How can we overcome the changes in the illumination ?

Adaptive thresholding

Adaptive Thresholding

Divide the image into sub-images.

Assume that the illumination in each sub-images is constant.

Use a different threshold for each sub-image.

Alternatively – use a running window (and use the threshold of the window only for the central pixel )


Rapid illumination changes.

Regions without text: we can try to recognize that these regions is unimodal.

Region based segmentation

Region-based segmentation

We would like to use spatial information.

We assume that neighboring pixels tend to belong to the same segment (not always true)

Edge detection: looking for the boundaries of the segments.

Problem: edges usually do not determine close contours. We can try to do it with edge linking (as in Canny’s edge detector)

Region based segmentation1

Region-based segmentation

Basic Formulation

Let R represent the entire image region.

Segmentation: Partitioning R into n subgroups Ri s.t:


b) is a connected region




P is the partition predicate

Region growing

Region growing

Choose a group of points as initial regions.

Expand the regions to neighboring pixels using a heuristic:

Color distance from the neighbors.

The total error in the region (till a certain threshold):


Sum of the differences between neighbors.

Maximal difference from a central pixel.

In some cases, we can also use structural information: the region size and shape.

In this way we can handle regions with a smoothly varying gray level or color.

Question: How do we choose the starting points ? It is less important if we also can merge regions.

Region merging and splitting

Region merging and splitting

In region merging we start with small regions (it can be pixels), and iteratively merge regions which are similar.

In region splitting, we start with the whole image, and split regions which are not uniform.

These methods can be combined. Formally:

Choose a predicate P.

Split into disjoint regions any region Ri for which

Merge any adjacent regions Ri and Rj for which

Stop when no further merging and splitting is possible.



















  • With quadtree, one can use a variation of the split & merge scheme:

    • Start with splitting regions.

    • Only at the final stage: merge regions.

Segmentation as clustering

Segmentation as clustering

  • Address the image as a set of points in the n-dimensional space:

    • Gray level images: p=(x,y,I(x,y)) in R3

    • Color images: p =(x,y,R(x,y),G(x,y),B(x,y)) in R5

    • Texture: p= (x,y,vector_of_fetures)

    • Color Histograms: p=(R(x,y),G(x,y),B(x,y)) in R3.we ignore the spatial information.

  • From this stage, we forget the meaning of each coordinate. We deal with arbitrary set of points.

  • Therefore, we first need to “normalize” the features (for example - convert a color image to the appropriate linear space representation)

Again we can use splitting merging

Again, we can use splitting & merging

Here, we merge each time the closest neighbors.



  • Idea:

    • Determine the number of clusters

    • •Find the cluster centers and point-cluster correspondences to minimize error

  • Problem: Exhaustive search is too expensive.

  • Solution: We will use instead an iterative search.[Recall the ideal quantization procedure.]


– fix cluster centers; allocate points to closest cluster

– fix allocation; compute best cluster centers

Error function =

Example clustering with k means using gray level and color histograms from slides by d a forsyth

Example – clustering with K-means using gray-level and color histograms(from slides by D.A. forsyth)

Mean shift

Mean Shift

  • K-means is a powerful and popular method for clustering. However:

    • It assumes a pre-determined number of clusters

    • It “likes” compact clusters. Sometimes, we are looking for long but continues clusters.

  • Mean Shift:

    • Determine a window size (usually small).

    • For each point p:

      • Compute a weighted mean of the shift in the window:

      • Set p := p + m

      • Continue until convergence.

    • At the end, use a more standard clustering method.

Mean shift cont

Mean Shift (cont’)

  • This method is based on the assumption that points are more and more dense as we are getting near the cluster “central mass”.

Motion segmentation

Motion segmentation

  • Background subtraction:

    • Assumes the existence of a dominant background.

  • Optical flow (use the motion vectors as features)

  • Multi model motion:

    • Divide the image to layers such that in each layer, there exist a parametric motion model.

  • Login