به نام خدا
Sponsored Links
This presentation is the property of its rightful owner.
1 / 91

By: Navid Einackchi PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

به نام خدا. Isfahan University of Technology Electrical and Computer Department. Master Thesis of Computer Engineering- Artificial Intelligence and Robotic. Image alignment and stitching using Object Recognition Methods. By: Navid Einackchi. Supervisors: Dr Rasoul Amirfattahi

Download Presentation

By: Navid Einackchi

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


به نام خدا

Isfahan University of Technology

Electrical and Computer Department

Master Thesis of Computer Engineering- Artificial Intelligence and Robotic

Image alignment and stitching using Object Recognition Methods

By: Navid Einackchi

Supervisors: Dr Rasoul Amirfattahi

Dr javad Askari

Advisor: Dr M. Saraee

Spring 2007


Presentation Process

  • Introduction

    • Definition

    • Aligning concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions

Some slides are from other sources


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


تصوير حاصل

Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Image Alignment and Stitching

  • Definition

  • Applications

    • Mosaic Image

    • Panorama

    • Virtual Environment

    • Robotic


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Aligning Concepts

  • Image Transformations

    • Applicable Transformations

  • Find transformations

    • How many parameters?

    • Calculating Parameters


Image Transformations

  • Transition

    • Number of Parameters: 2

    • Number of Points: 1


Image Transformations

  • Euclidean

    • Number of Parameters: 3

    • Number of Points: 2


Image Transformations

  • Similarity

    • Number of Parameters: 4

    • Number of Points: 2


Image Transformations

  • Affine

    • Number of Parameters: 6

    • Number of Points: 3


Image Transformations

  • Perspective

    • Number of Parameters: 8

    • Number of Points: 4


Image Transformation Computation

  • Direct

    • Mapping one image into another using different parameters

    • Define an Error Function based on pixel intensity difference

  • Using Corresponding points

    • Obtaining Corresponding points

    • Computing Transformation parameters using corresponding points


Direct Method

Error Function


Direct Method

Error Function

؟


Direct Method


Using Corresponding points


Using Corresponding points


Using Corresponding points


Using Corresponding points

  • How many points?

  • How to select?

    • Using human

    • Automatically


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching

  • Stitching

  • Choices

    • Final Plane

    • Pixel weighting


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Object Recognition Methods

  • Local Feature Detection

    • Edge

    • Corner

    • Hole

  • Interest Regions Matching

    • Appearance Matching

    • Geometric Matching

  • Finding Object Relations in Image


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Salient Features

  • Features of Feature!

    • Invariants against transformations

    • Ability to explain the image

    • Ability to being Matched (Repeatability)

  • Selected Features

    • Corners

    • Holes


Harris Detector

Plain

No changes in every direction

Edge

High changes in Edge Direction

Corner

High changes in every direction


Harris Detector(Continued)

Fast change Direction

(max)-1/2

Slow change direction

(min)-1/2


Harris Detector(Continued)

Edge2 >> 1

2

“Corner”1,2both big1 ~ 2

Edge1 >> 2

Plain

1


Harris Detector(Continued)

  • Harris measurement


Harris Detector(Continued)

  • Harris measurement response

2

“Edge”

R < 0

“Corner”

R > 0

Local Maximum bigger than Threshold

“Plain”

“Edge”

|R| small

R < 0

1


Threshold

R

R

x

x

Harris Detector(Continued)

  • Features

    • Invariant against rotation

    • Invariant against intensity


Harris Detector(Continued)

  • Example


Scale Invariant

  • Scale Problem


Scale Invariant

Finding Appropriate Window Size


-

=

Scale Selection

  • Using Differential of Gaussian Filters

    • Laplacian of Gaussian (LoG)

    • Difference of Gaussian (DoG)

Window thst maximize filter


Automatic Scale Selection


Automatic Scale Selection


مقیاس

 DoG 

y

x

 DoG 

SIFT Detector

  • Bulb Detector

  • Invariant against scale

  • Using DoG

    • Bulb detection

    • Scale detection


Bulb position

Scale position

SIFT Detector

  • Detecting Bulb and Scale Simultaneously

    • Bulb position = maximum in plain

    • Scale = maximum in third dimension


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Descriptor

  • Vector which describes region of interest point

    • Pixels

    • Histogram

    • Differential

  • Feature of Descriptor

    • Invariant against image distortion (view, angle, intensity)

    • Able to express similarities and differences


2p

0

SIFT Descriptor

  • Based on value and direction of gradients

  • 128 dimensional Vector

  • Construction

    • Calculation of value and direction of gradient of each pixel

    • Making direction histogram of gradients

    • Rotating interest region based on dominant direction

    • Dividing the region into 16 region

    • Constructing direction histogram for each region


Construction Example


DescriptorFeatures

  • Invariant against

    • Rotation

    • Position

    • Affine changes in intensity(I  Ia+ b)


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Matching

  • Appearance Matching

    • Comparing distance of each two descriptors

    • Euclidean Distance

    • K nearest neighbor

  • Geometrical Matching

    • Based on geometrical relations between points

    • Estimation of image transformation

    • Rejecting false corresponding points


Example of Geometrical Matching


Automatically image alignment

  • Direct Method

  • Based on interest points


Compare


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Brown and Lowe Algorithm

O(n4 m4)


Brown and Lowe Algorithm


Brown and Lowe Algorithm


Similarity Transform rather than perspective

  • Why

    • Detector are unable to handle perspective

    • Small portion of each images is overlapped

    • Increasing the tolerance of true corresponding

  • So We use Similarity Transformation


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations


First Proposed Algorithm

  • Detecting Interest points using Harris detector

  • Defining descriptors for each point

  • Finding corresponding points using appearance

    • Comparing interest points with each other

    • K Nearest Neighbor

  • Finding relation between 2 images

    • Using Similarity

    • Based on voting

  • Rejecting wrong corresponding points based on geometrical relations


Voting Mechanism


Voting Mechanism

Angle

Scale

3469

2255

از مرتبهO(n2 m2)


Applied Rotation

Applied Scale

Estimated Scale

Estimated Rotation

Evaluation


Concluding

  • Very good for rotation

  • Good for little scaling

  • Weak for big scaling

    • Due to Harris detector weakness


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Improvement

  • Using less time order algorithm

  • Using Scale Invariant Feature Transform


Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Distance Ration Threshold

  • Find 1NN and its distance = D1

  • Find 2NN and its distance = D2

  • Find ratio of these distances (D1/D2 = ratio)

    • If Lower than threshold -> reject

    • Else accept


Distance Ration ThresholdMy Improvement

  • Find 1NN and its distance = D1

  • Multiply D1 by ratio = range

  • Look for range in k-d tree


Results of threshold on ratio

  • Threshold Ratio = 0.5


Results of threshold on ratio

  • Threshold Ratio = 0.7


Second proposed Algorithm

  • Detecting Interest points

    • SIFT Detector

  • Finding Corresponding points

    • Based on appearance

    • Using k-d tree

    • Distance ratio threshold

  • Finding relation between images

    • Voting on step 2 based on Appearance corresponding

  • Rejecting wrong corresponding points

    • Based on geometric relations between points


Estimating Geometrical Relation

  • Each interest point has

    • Dominant Direction

    • Scale

  • Use this information for voting


Estimation Histogram

Direction

Direction

Scale

Scale

O(nm)

O(n)

Maximum

Maximum

Using 3NN method

Using threshold method


Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing Knn and Ratio Threshold settingBased on Number of true corresponding points


Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Comparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)


Applied Scale

Applied Rotation

Estimated Rotation

Estimated Scale

Evaluation


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Stitching

  • Pixel Weights based on their distance


Stitching


Stitching


Other examples

  • Adobe Photoshop could not stitch these images


Other Examples


Other Examples


Presentation Process

  • Introduction

    • Definition

    • Aligning Concepts

    • Stitching Concepts

  • Object Recognition Problem

    • Salient Feature

    • What is Descriptor

    • Matching

  • Proposed Method for Aligning

    • Using Harris Detector

    • Using SIFT Detector

    • Stitching

  • Concluding and Suggestions


Concluding

  • Ideas

    • Using Harris and SIFT as a detector

    • Using Similarity transform as a general image transform

    • Using voting based on direction and scale of each interest points

    • Introducing Evaluation Method

  • Results

    • Harris is more reliable but SIFT is much better in presence of scale transform

    • Similarity can handle prospective transform for human eyes

    • Voting can obtain similarity transform with much lower time order

    • Threshold ratio has a better behavior than Knn


Thanks

Questions?


  • Login