به نام خدا
Download
1 / 91

By: Navid Einackchi - PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on

به نام خدا. Isfahan University of Technology Electrical and Computer Department. Master Thesis of Computer Engineering- Artificial Intelligence and Robotic. Image alignment and stitching using Object Recognition Methods. By: Navid Einackchi. Supervisors: Dr Rasoul Amirfattahi

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' By: Navid Einackchi' - hall


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

به نام خدا

Isfahan University of Technology

Electrical and Computer Department

Master Thesis of Computer Engineering- Artificial Intelligence and Robotic

Image alignment and stitching using Object Recognition Methods

By: Navid Einackchi

Supervisors: Dr Rasoul Amirfattahi

Dr javad Askari

Advisor: Dr M. Saraee

Spring 2007


Presentation process
Presentation Process

  • Introduction

    • Definition

    • Aligning concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions

Some slides are from other sources


Presentation process1
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Presentation process2
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Image alignment and stitching

تصوير حاصل

Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image alignment and stitching1
Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image alignment and stitching2
Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Presentation process3
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Aligning concepts
Aligning Concepts

  • Image Transformations

    • Applicable Transformations

  • Find transformations

    • How many parameters?

    • Calculating Parameters


Image transformations
Image Transformations

  • Transition

    • Number of Parameters: 2

    • Number of Points: 1


Image transformations1
Image Transformations

  • Euclidean

    • Number of Parameters: 3

    • Number of Points: 2


Image transformations2
Image Transformations

  • Similarity

    • Number of Parameters: 4

    • Number of Points: 2


Image transformations3
Image Transformations

  • Affine

    • Number of Parameters: 6

    • Number of Points: 3


Image transformations4
Image Transformations

  • Perspective

    • Number of Parameters: 8

    • Number of Points: 4


Image transformation computation
Image Transformation Computation

  • Direct

    • Mapping one image into another using different parameters

    • Define an Error Function based on pixel intensity difference

  • Using Corresponding points

    • Obtaining Corresponding points

    • Computing Transformation parameters using corresponding points


Direct method
Direct Method

Error Function


Direct method1
Direct Method

Error Function

؟






Using Corresponding points

  • How many points?

  • How to select?

    • Using human

    • Automatically


Presentation process4
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching
Stitching

  • Stitching

  • Choices

    • Final Plane

    • Pixel weighting


Presentation process5
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Object recognition methods
Object Recognition Methods

  • Local Feature Detection

    • Edge

    • Corner

    • Hole

  • Interest Regions Matching

    • Appearance Matching

    • Geometric Matching

  • Finding Object Relations in Image


Presentation process6
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Salient features
Salient Features

  • Features of Feature!

    • Invariants against transformations

    • Ability to explain the image

    • Ability to being Matched (Repeatability)

  • Selected Features

    • Corners

    • Holes


Harris detector
Harris Detector

Plain

No changes in every direction

Edge

High changes in Edge Direction

Corner

High changes in every direction


Harris detector continued
Harris Detector(Continued)

Fast change Direction

(max)-1/2

Slow change direction

(min)-1/2


Harris detector continued1
Harris Detector(Continued)

Edge2 >> 1

2

“Corner”1,2both big1 ~ 2

Edge1 >> 2

Plain

1


Harris detector continued2
Harris Detector(Continued)

  • Harris measurement


Harris detector continued3
Harris Detector(Continued)

  • Harris measurement response

2

“Edge”

R < 0

“Corner”

R > 0

Local Maximum bigger than Threshold

“Plain”

“Edge”

|R| small

R < 0

1


Harris detector continued4

Threshold

R

R

x

x

Harris Detector(Continued)

  • Features

    • Invariant against rotation

    • Invariant against intensity


Harris detector continued5
Harris Detector(Continued)

  • Example


Scale Invariant

  • Scale Problem


Scale invariant
Scale Invariant

Finding Appropriate Window Size


Scale selection

-

=

Scale Selection

  • Using Differential of Gaussian Filters

    • Laplacian of Gaussian (LoG)

    • Difference of Gaussian (DoG)

Window thst maximize filter




Sift detector

مقیاس

 DoG 

y

x

 DoG 

SIFT Detector

  • Bulb Detector

  • Invariant against scale

  • Using DoG

    • Bulb detection

    • Scale detection


Sift detector1

Bulb position

Scale position

SIFT Detector

  • Detecting Bulb and Scale Simultaneously

    • Bulb position = maximum in plain

    • Scale = maximum in third dimension


Presentation process7
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Descriptor
Descriptor

  • Vector which describes region of interest point

    • Pixels

    • Histogram

    • Differential

  • Feature of Descriptor

    • Invariant against image distortion (view, angle, intensity)

    • Able to express similarities and differences


Sift descriptor

2p

0

SIFT Descriptor

  • Based on value and direction of gradients

  • 128 dimensional Vector

  • Construction

    • Calculation of value and direction of gradient of each pixel

    • Making direction histogram of gradients

    • Rotating interest region based on dominant direction

    • Dividing the region into 16 region

    • Constructing direction histogram for each region



Descriptor features
DescriptorFeatures

  • Invariant against

    • Rotation

    • Position

    • Affine changes in intensity(I  Ia+ b)


Presentation process8
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Matching
Matching

  • Appearance Matching

    • Comparing distance of each two descriptors

    • Euclidean Distance

    • K nearest neighbor

  • Geometrical Matching

    • Based on geometrical relations between points

    • Estimation of image transformation

    • Rejecting false corresponding points



Automatically image alignment
Automatically image alignment

  • Direct Method

  • Based on interest points



Presentation process9
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions





Similarity transform rather than perspective
Similarity Transform rather than perspective

  • Why

    • Detector are unable to handle perspective

    • Small portion of each images is overlapped

    • Increasing the tolerance of true corresponding

  • So We use Similarity Transformation


Presentation process10
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


First proposed algorithm
First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations


First proposed algorithm1
First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations



Voting mechanism1
Voting Mechanism

Angle

Scale

3469

2255

از مرتبهO(n2 m2)


Evaluation

Applied Rotation

Applied Scale

Estimated Scale

Estimated Rotation

Evaluation


Concluding
Concluding

  • Very good for rotation

  • Good for little scaling

  • Weak for big scaling

    • Due to Harris detector weakness


Presentation process11
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Improvement
Improvement

  • Using less time order algorithm

  • Using Scale Invariant Feature Transform


Second proposed algorithm
Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Distance ration threshold
Distance Ration Threshold

  • Find 1NN and its distance = D1

  • Find 2NN and its distance = D2

  • Find ratio of these distances (D1/D2 = ratio)

    • If Lower than threshold -> reject

    • Else accept


Distance ration threshold my improvement
Distance Ration ThresholdMy Improvement

  • Find 1NN and its distance = D1

  • Multiply D1 by ratio = range

  • Look for range in k-d tree


Results of threshold on ratio
Results of threshold on ratio

  • Threshold Ratio = 0.5


Results of threshold on ratio1
Results of threshold on ratio

  • Threshold Ratio = 0.7


Second proposed algorithm1
Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Estimating geometrical relation
Estimating Geometrical Relation

  • Each interest point has

    • Dominant Direction

    • Scale

  • Use this information for voting


Estimation histogram
Estimation Histogram

Direction

Direction

Scale

Scale

O(nm)

O(n)

Maximum

Maximum

Using 3NN method

Using threshold method


Comparing knn and ratio threshold setting based on number of true corresponding points
Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing knn and ratio threshold setting based on number of true corresponding points1
Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing knn and ratio threshold setting in tolerating false corresponding
Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Comparing knn and ratio threshold setting in tolerating false corresponding1
Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Evaluation1

Applied Scale

Applied Rotation

Estimated Rotation

Estimated Scale

Evaluation


Presentation process12
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching1
Stitching

  • Pixel Weights based on their distance




Other examples
Other examples

  • Adobe Photoshop could not stitch these images




Presentation process13
Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Concluding1
Concluding

  • Ideas

    • Using Harris and SIFT as a detector

    • Using Similarity transform as a general image transform

    • Using voting based on direction and scale of each interest points

    • Introducing Evaluation Method

  • Results

    • Harris is more reliable but SIFT is much better in presence of scale transform

    • Similarity can handle prospective transform for human eyes

    • Voting can obtain similarity transform with much lower time order

    • Threshold ratio has a better behavior than Knn


Thanks

Questions?