Vision based motion control of robots
Download
1 / 40

Vision-Based Motion Control of Robots - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

Vision-Based Motion Control of Robots. Azad Shademan Guest Lecturer CMPUT 412 – Experimental Robotics Computing Science, University of Alberta Edmonton, Alberta, CANADA. desired. Left Image. Right Image. current. Vision-Based Control. B. B. A. A. A. B. Vision-Based Control.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Vision-Based Motion Control of Robots' - ulric


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Vision based motion control of robots

Vision-Based Motion Control of Robots

Azad Shademan

Guest Lecturer

CMPUT 412 – Experimental Robotics

Computing Science, University of Alberta

Edmonton, Alberta, CANADA


Vision based control

desired

Left Image

Right Image

current

Vision-Based Control

B

B

A

A

A

B

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based control1
Vision-Based Control

Left Image

Right Image

B

B

B

A. Shademan. CMPUT 412, Vision-based motion control of robots


Vision based control2
Vision-Based Control

  • Feedback from visual sensor (camera) to control a robot

  • Also called “Visual Servoing”

  • Is it any difficult?

    Images are 2D, the robot workspace is 3D 2D data  3D geometry

A. Shademan. CMPUT 412, Vision-based motion control of robots


Where is the camera located

Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand

Where is the camera located?

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual servo control law
Visual Servo Control law

  • Position-Based:

    • Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

  • Image-Based:

    • Desired image features seen from camera

    • Control law entirely based on image features

A. Shademan. CMPUT 412, Vision-based motion control of robots


Position based
Position-Based

Desired pose

Estimated pose

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based
Image-Based

Desired

Image feature

Extracted

image feature

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual motor equation

x1

x3

x4

x2

q=[q1 … q6]

This Jacobian is important

for motion control.

Visual-motor Equation

Visual-Motor Equation

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual motor jacobian

B

A

A

B

Visual-motor Jacobian

Joint space

velocity

Image space

velocity

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based control law
Image-Based Control Law

  • Measure the error in image space

  • Calculate/Estimate the inverse Jacobian

  • Update new joint values

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image based control law1

Desired

Image feature

Extracted

image feature

Image-Based Control Law

A. Shademan. CMPUT 412, Vision-based motion control of robots


Jacobian calculation
Jacobian calculation

  • Analytic form available if model is known. Known model  Calibrated

  • Must be estimated if model is not known

    Unknown model  Uncalibrated

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image jacobian calibrated
Image Jacobian (calibrated)

  • Analytic form depends on depth estimates.

Camera

Velocity

  • Camera/Robot transform required.

  • No flexibility.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Image jacobian uncalibrated
Image Jacobian (uncalibrated)

  • A popular local estimator:

    Recursive secant method (Broyden update):

A. Shademan. CMPUT 412, Vision-based motion control of robots


Calibrated vs uncalibrated

Relaxed model assumptions

Traditionally:

Local methods

No global planning 

Difficult to show asymptotic stability condition is ensured 

The problem of traditional methods is the locality.

Model derived analytically

Global asymptotic stability 

Optimal planning is

possible 

A lot of prior knowledge on the model 

Calibrated vs. Uncalibrated

  • Global Model Estimation (Research result)

    • Optimal trajectory planning 

    • Global stability guarantee 

A. Shademan. CMPUT 412, Vision-based motion control of robots


Synopsis of global visual servoing
Synopsis of Global Visual Servoing

  • Model Estimation (Uncalibrated)

  • Visual-Motor Kinematics Model

  • Global Model

    • Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

  • Our contributions:

    • K-NN Regression-Based Estimation

    • Locally Least Squares Estimation

A. Shademan. CMPUT 412, Vision-based motion control of robots


Local vs global

Key idea: using only the previous estimation to estimate the Jacobian

RLS with forgetting factor Hosoda and Asada ’94

1st Rank Broyden update: Jägersand et al. ’97

Exploratory motion: Sutanto et al. ‘98

Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning

Local methods don’t!

Local vs. Global

A. Shademan. CMPUT 412, Vision-based motion control of robots


K nn regression based method

x1

?

3 NN

q2

q1

K-NN Regression-based Method

q2

q1

A. Shademan. CMPUT 412, Vision-based motion control of robots


x1

?

q2

KNN(q)

q1

Locally Least Squares Method

(X,q)

A. Shademan. CMPUT 412, Vision-based motion control of robots


Experimental setup
Experimental Setup

  • Puma 560

  • Eye-to-hand configuration

  • Stereo vision

  • Features: projection of the end-effector’s position on image planes (4-dim)

  • 3 DOF for control

A. Shademan. CMPUT 412, Vision-based motion control of robots


Measuring the estimation error
Measuring the Estimation Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Global estimation error
Global Estimation Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Noise on estimation quality

With increasing noise level, the error decreases

Noise on Estimation Quality

KNN

LLS

A. Shademan. CMPUT 412, Vision-based motion control of robots


Effect of number of neighbors
Effect of Number of Neighbors

A. Shademan. CMPUT 412, Vision-based motion control of robots


Conclusions

Presented two global methods to learn the visual-motor function

LLS (global) works better than the KNN (global) and local updates.

KNN suffers from the bias in local estimations

Noise helps system identification

Conclusions

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator
Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator1
Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator2
Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Eye in hand simulator3
Eye-in-Hand Simulator

A. Shademan. CMPUT 412, Vision-based motion control of robots


Mean squared error
Mean-Squared-Error

A. Shademan. CMPUT 412, Vision-based motion control of robots


Task errors
Task Errors

A. Shademan. CMPUT 412, Vision-based motion control of robots


Questions
Questions?

A. Shademan. CMPUT 412, Vision-based motion control of robots


Position based1
Position-Based

  • Robust and real-time relative pose estimation

  • Extended Kalman Filter to solve the nonlinear relative pose equations.

  • Cons:

    • EKF is not the optimal estimator.

    • Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Overview of pbvs

What kind of nonlinearity?

IEKF

Overview of PBVS

2D-3D nonlinear point correspondences

T. Lefebvre et al. “Kalman Filters for Nonlinear Systems: A Comparison of Performance,” Intl. J. of Control, vol. 77, no. 7, pp. 639-653, May 2004.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Ekf pose estimation
EKF Pose Estimation

yaw

pitch

roll

State variable

Process noise

Measurement noise

Measurement equation is nonlinear and must be linearized.

A. Shademan. CMPUT 412, Vision-based motion control of robots


Visual servoing based on the estimated global model
Visual-Servoing Based on the Estimated Global Model

A. Shademan. CMPUT 412, Vision-based motion control of robots


Control based on local models
Control Based on Local Models

A. Shademan. CMPUT 412, Vision-based motion control of robots


Estimation for local methods
Estimation for Local Methods

  • In practice: Broyden 1st-rank estimation, RLS with forgetting factor, etc.

A. Shademan. CMPUT 412, Vision-based motion control of robots


A. Shademan. CMPUT 412, Vision-based motion control of robots


ad