Vision based motion control of robots
This presentation is the property of its rightful owner.
Sponsored Links
1 / 40

Vision-Based Motion Control of Robots PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on
  • Presentation posted in: General

Vision-Based Motion Control of Robots. Azad Shademan Guest Lecturer CMPUT 412 – Experimental Robotics Computing Science, University of Alberta Edmonton, Alberta, CANADA. desired. Left Image. Right Image. current. Vision-Based Control. B. B. A. A. A. B. Vision-Based Control.

Download Presentation

Vision-Based Motion Control of Robots

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Vision based motion control of robots

Vision-Based Motion Control of Robots

Azad Shademan

Guest Lecturer

CMPUT 412 – Experimental Robotics

Computing Science, University of Alberta

Edmonton, Alberta, CANADA


Vision based control

desired

Left Image

Right Image

current

Vision-Based Control

B

B

A

A

A

B

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based control1

Vision-Based Control

Left Image

Right Image

B

B

B

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based control2

Vision-Based Control

  • Feedback from visual sensor (camera) to control a robot

  • Also called “Visual Servoing”

  • Is it any difficult?

    Images are 2D, the robot workspace is 3D 2D data  3D geometry

A. Shademan. CMPUT 412, Vision-based motion control of robots


Where is the camera located

Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand

Where is the camera located?

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual servo control law

Visual Servo Control law

  • Position-Based:

    • Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

  • Image-Based:

    • Desired image features seen from camera

    • Control law entirely based on image features

A. Shademan. CMPUT 412, Vision-based motion control of robots


Position based

Position-Based

Desired pose

Estimated pose

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based

Image-Based

Desired

Image feature

Extracted

image feature

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual motor equation

x1

x3

x4

x2

q=[q1 … q6]

This Jacobian is important

for motion control.

Visual-motor Equation

Visual-Motor Equation

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual motor jacobian

B

A

A

B

Visual-motor Jacobian

Joint space

velocity

Image space

velocity

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based control law

Image-Based Control Law

  • Measure the error in image space

  • Calculate/Estimate the inverse Jacobian

  • Update new joint values

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based control law1

Desired

Image feature

Extracted

image feature

Image-Based Control Law

A. Shademan. CMPUT 412, Vision-based motion control of robots


Jacobian calculation

Jacobian calculation

  • Analytic form available if model is known. Known model  Calibrated

  • Must be estimated if model is not known

    Unknown model  Uncalibrated

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image jacobian calibrated

Image Jacobian (calibrated)

  • Analytic form depends on depth estimates.

Camera

Velocity

  • Camera/Robot transform required.

  • No flexibility.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image jacobian uncalibrated

Image Jacobian (uncalibrated)

  • A popular local estimator:

    Recursive secant method (Broyden update):

A. Shademan. CMPUT 412, Vision-based motion control of robots


Calibrated vs uncalibrated

Relaxed model assumptions

Traditionally:

Local methods

No global planning 

Difficult to show asymptotic stability condition is ensured 

The problem of traditional methods is the locality.

Model derived analytically

Global asymptotic stability 

Optimal planning is

possible 

A lot of prior knowledge on the model 

Calibrated vs. Uncalibrated

  • Global Model Estimation (Research result)

    • Optimal trajectory planning 

    • Global stability guarantee 

A. Shademan. CMPUT 412, Vision-based motion control of robots


Synopsis of global visual servoing

Synopsis of Global Visual Servoing

  • Model Estimation (Uncalibrated)

  • Visual-Motor Kinematics Model

  • Global Model

    • Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

  • Our contributions:

    • K-NN Regression-Based Estimation

    • Locally Least Squares Estimation

A. Shademan. CMPUT 412, Vision-based motion control of robots


Local vs global

Key idea: using only the previous estimation to estimate the Jacobian

RLS with forgetting factor Hosoda and Asada ’94

1st Rank Broyden update: Jägersand et al. ’97

Exploratory motion: Sutanto et al. ‘98

Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning

Local methods don’t!

Local vs. Global

A. Shademan. CMPUT 412, Vision-based motion control of robots


K nn regression based method

x1

?

3 NN

q2

q1

K-NN Regression-based Method

q2

q1

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based motion control of robots

x1

?

q2

KNN(q)

q1

Locally Least Squares Method

(X,q)

A. Shademan. CMPUT 412, Vision-based motion control of robots


Experimental setup

Experimental Setup

  • Puma 560

  • Eye-to-hand configuration

  • Stereo vision

  • Features: projection of the end-effector’s position on image planes (4-dim)

  • 3 DOF for control

A. Shademan. CMPUT 412, Vision-based motion control of robots


Measuring the estimation error

Measuring the Estimation Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Global estimation error

Global Estimation Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Noise on estimation quality

With increasing noise level, the error decreases

Noise on Estimation Quality

KNN

LLS

A. Shademan. CMPUT 412, Vision-based motion control of robots


Effect of number of neighbors

Effect of Number of Neighbors

A. Shademan. CMPUT 412, Vision-based motion control of robots


Conclusions

Presented two global methods to learn the visual-motor function

LLS (global) works better than the KNN (global) and local updates.

KNN suffers from the bias in local estimations

Noise helps system identification

Conclusions

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator

Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator1

Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator2

Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator3

Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Mean squared error

Mean-Squared-Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Task errors

Task Errors

A. Shademan. CMPUT 412, Vision-based motion control of robots


Questions

Questions?

A. Shademan. CMPUT 412, Vision-based motion control of robots


Position based1

Position-Based

  • Robust and real-time relative pose estimation

  • Extended Kalman Filter to solve the nonlinear relative pose equations.

  • Cons:

    • EKF is not the optimal estimator.

    • Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Overview of pbvs

What kind of nonlinearity?

IEKF

Overview of PBVS

2D-3D nonlinear point correspondences

T. Lefebvre et al. “Kalman Filters for Nonlinear Systems: A Comparison of Performance,” Intl. J. of Control, vol. 77, no. 7, pp. 639-653, May 2004.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Ekf pose estimation

EKF Pose Estimation

yaw

pitch

roll

State variable

Process noise

Measurement noise

Measurement equation is nonlinear and must be linearized.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual servoing based on the estimated global model

Visual-Servoing Based on the Estimated Global Model

A. Shademan. CMPUT 412, Vision-based motion control of robots


Control based on local models

Control Based on Local Models

A. Shademan. CMPUT 412, Vision-based motion control of robots


Estimation for local methods

Estimation for Local Methods

  • In practice: Broyden 1st-rank estimation, RLS with forgetting factor, etc.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based motion control of robots

A. Shademan. CMPUT 412, Vision-based motion control of robots


  • Login