به نام خدا
This presentation is the property of its rightful owner.
Sponsored Links
1 / 91

By: Navid Einackchi PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

به نام خدا. Isfahan University of Technology Electrical and Computer Department. Master Thesis of Computer Engineering- Artificial Intelligence and Robotic. Image alignment and stitching using Object Recognition Methods. By: Navid Einackchi. Supervisors: Dr Rasoul Amirfattahi

Download Presentation

By: Navid Einackchi

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


By navid einackchi

به نام خدا

Isfahan University of Technology

Electrical and Computer Department

Master Thesis of Computer Engineering- Artificial Intelligence and Robotic

Image alignment and stitching using Object Recognition Methods

By: Navid Einackchi

Supervisors: Dr Rasoul Amirfattahi

Dr javad Askari

Advisor: Dr M. Saraee

Spring 2007


Presentation process

Presentation Process

  • Introduction

    • Definition

    • Aligning concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions

Some slides are from other sources


Presentation process1

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Presentation process2

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Image alignment and stitching

تصوير حاصل

Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image alignment and stitching1

Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image alignment and stitching2

Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Presentation process3

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Aligning concepts

Aligning Concepts

  • Image Transformations

    • Applicable Transformations

  • Find transformations

    • How many parameters?

    • Calculating Parameters


Image transformations

Image Transformations

  • Transition

    • Number of Parameters: 2

    • Number of Points: 1


Image transformations1

Image Transformations

  • Euclidean

    • Number of Parameters: 3

    • Number of Points: 2


Image transformations2

Image Transformations

  • Similarity

    • Number of Parameters: 4

    • Number of Points: 2


Image transformations3

Image Transformations

  • Affine

    • Number of Parameters: 6

    • Number of Points: 3


Image transformations4

Image Transformations

  • Perspective

    • Number of Parameters: 8

    • Number of Points: 4


Image transformation computation

Image Transformation Computation

  • Direct

    • Mapping one image into another using different parameters

    • Define an Error Function based on pixel intensity difference

  • Using Corresponding points

    • Obtaining Corresponding points

    • Computing Transformation parameters using corresponding points


Direct method

Direct Method

Error Function


Direct method1

Direct Method

Error Function

؟


Direct method2

Direct Method


Using corresponding points

Using Corresponding points


Using corresponding points1

Using Corresponding points


Using corresponding points2

Using Corresponding points


By navid einackchi

Using Corresponding points

  • How many points?

  • How to select?

    • Using human

    • Automatically


Presentation process4

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching

Stitching

  • Stitching

  • Choices

    • Final Plane

    • Pixel weighting


Presentation process5

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Object recognition methods

Object Recognition Methods

  • Local Feature Detection

    • Edge

    • Corner

    • Hole

  • Interest Regions Matching

    • Appearance Matching

    • Geometric Matching

  • Finding Object Relations in Image


Presentation process6

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Salient features

Salient Features

  • Features of Feature!

    • Invariants against transformations

    • Ability to explain the image

    • Ability to being Matched (Repeatability)

  • Selected Features

    • Corners

    • Holes


Harris detector

Harris Detector

Plain

No changes in every direction

Edge

High changes in Edge Direction

Corner

High changes in every direction


Harris detector continued

Harris Detector(Continued)

Fast change Direction

(max)-1/2

Slow change direction

(min)-1/2


Harris detector continued1

Harris Detector(Continued)

Edge2 >> 1

2

“Corner”1,2both big1 ~ 2

Edge1 >> 2

Plain

1


Harris detector continued2

Harris Detector(Continued)

  • Harris measurement


Harris detector continued3

Harris Detector(Continued)

  • Harris measurement response

2

“Edge”

R < 0

“Corner”

R > 0

Local Maximum bigger than Threshold

“Plain”

“Edge”

|R| small

R < 0

1


Harris detector continued4

Threshold

R

R

x

x

Harris Detector(Continued)

  • Features

    • Invariant against rotation

    • Invariant against intensity


Harris detector continued5

Harris Detector(Continued)

  • Example


By navid einackchi

Scale Invariant

  • Scale Problem


Scale invariant

Scale Invariant

Finding Appropriate Window Size


Scale selection

-

=

Scale Selection

  • Using Differential of Gaussian Filters

    • Laplacian of Gaussian (LoG)

    • Difference of Gaussian (DoG)

Window thst maximize filter


Automatic scale selection

Automatic Scale Selection


Automatic scale selection1

Automatic Scale Selection


Sift detector

مقیاس

 DoG 

y

x

 DoG 

SIFT Detector

  • Bulb Detector

  • Invariant against scale

  • Using DoG

    • Bulb detection

    • Scale detection


Sift detector1

Bulb position

Scale position

SIFT Detector

  • Detecting Bulb and Scale Simultaneously

    • Bulb position = maximum in plain

    • Scale = maximum in third dimension


Presentation process7

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Descriptor

Descriptor

  • Vector which describes region of interest point

    • Pixels

    • Histogram

    • Differential

  • Feature of Descriptor

    • Invariant against image distortion (view, angle, intensity)

    • Able to express similarities and differences


Sift descriptor

2p

0

SIFT Descriptor

  • Based on value and direction of gradients

  • 128 dimensional Vector

  • Construction

    • Calculation of value and direction of gradient of each pixel

    • Making direction histogram of gradients

    • Rotating interest region based on dominant direction

    • Dividing the region into 16 region

    • Constructing direction histogram for each region


Construction example

Construction Example


Descriptor features

DescriptorFeatures

  • Invariant against

    • Rotation

    • Position

    • Affine changes in intensity(I  Ia+ b)


Presentation process8

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Matching

Matching

  • Appearance Matching

    • Comparing distance of each two descriptors

    • Euclidean Distance

    • K nearest neighbor

  • Geometrical Matching

    • Based on geometrical relations between points

    • Estimation of image transformation

    • Rejecting false corresponding points


Example of geometrical matching

Example of Geometrical Matching


Automatically image alignment

Automatically image alignment

  • Direct Method

  • Based on interest points


Compare

Compare


Presentation process9

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Brown and lowe algorithm

Brown and Lowe Algorithm

O(n4 m4)


Brown and lowe algorithm1

Brown and Lowe Algorithm


Brown and lowe algorithm2

Brown and Lowe Algorithm


Similarity transform rather than perspective

Similarity Transform rather than perspective

  • Why

    • Detector are unable to handle perspective

    • Small portion of each images is overlapped

    • Increasing the tolerance of true corresponding

  • So We use Similarity Transformation


Presentation process10

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


First proposed algorithm

First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations


First proposed algorithm1

First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations


Voting mechanism

Voting Mechanism


Voting mechanism1

Voting Mechanism

Angle

Scale

3469

2255

از مرتبهO(n2 m2)


Evaluation

Applied Rotation

Applied Scale

Estimated Scale

Estimated Rotation

Evaluation


Concluding

Concluding

  • Very good for rotation

  • Good for little scaling

  • Weak for big scaling

    • Due to Harris detector weakness


Presentation process11

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Improvement

Improvement

  • Using less time order algorithm

  • Using Scale Invariant Feature Transform


Second proposed algorithm

Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Distance ration threshold

Distance Ration Threshold

  • Find 1NN and its distance = D1

  • Find 2NN and its distance = D2

  • Find ratio of these distances (D1/D2 = ratio)

    • If Lower than threshold -> reject

    • Else accept


Distance ration threshold my improvement

Distance Ration ThresholdMy Improvement

  • Find 1NN and its distance = D1

  • Multiply D1 by ratio = range

  • Look for range in k-d tree


Results of threshold on ratio

Results of threshold on ratio

  • Threshold Ratio = 0.5


Results of threshold on ratio1

Results of threshold on ratio

  • Threshold Ratio = 0.7


Second proposed algorithm1

Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Estimating geometrical relation

Estimating Geometrical Relation

  • Each interest point has

    • Dominant Direction

    • Scale

  • Use this information for voting


Estimation histogram

Estimation Histogram

Direction

Direction

Scale

Scale

O(nm)

O(n)

Maximum

Maximum

Using 3NN method

Using threshold method


Comparing knn and ratio threshold setting based on number of true corresponding points

Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing knn and ratio threshold setting based on number of true corresponding points1

Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing knn and ratio threshold setting in tolerating false corresponding

Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Comparing knn and ratio threshold setting in tolerating false corresponding1

Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Evaluation1

Applied Scale

Applied Rotation

Estimated Rotation

Estimated Scale

Evaluation


Presentation process12

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching1

Stitching

  • Pixel Weights based on their distance


Stitching2

Stitching


Stitching3

Stitching


Other examples

Other examples

  • Adobe Photoshop could not stitch these images


Other examples1

Other Examples


Other examples2

Other Examples


Presentation process13

Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Concluding1

Concluding

  • Ideas

    • Using Harris and SIFT as a detector

    • Using Similarity transform as a general image transform

    • Using voting based on direction and scale of each interest points

    • Introducing Evaluation Method

  • Results

    • Harris is more reliable but SIFT is much better in presence of scale transform

    • Similarity can handle prospective transform for human eyes

    • Voting can obtain similarity transform with much lower time order

    • Threshold ratio has a better behavior than Knn


By navid einackchi

Thanks

Questions?


  • Login