1 / 63

Enhanced Camera Calibration Strategies: Theory & Implementation

Explore active calibration techniques for precise camera calibration. Learn theoretical derivation, strategies, error analysis, and experimental results. Find out how active calibration differs and enables accurate localization of contours without prior information.

liseli
Download Presentation

Enhanced Camera Calibration Strategies: Theory & Implementation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Active Calibration of Cameras:Theory and ImplementationAnupBasu Sung Huh CPSC 643 Individual Presentation II March 4th, 2009

  2. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  3. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  4. Introduction • Important step for a 2D image to relates to the 3D world • Involves relating the optical features of a lens to the sensing device • Pose estimation, 3D motion estimation, automated assembly • Parameters: image center and focal length • Expressed in terms of image pixels • Linear vs. Nonlinear, Lens distortion consideration vs. w/o consideration

  5. Linear Nonlinear • Simpler to implement • Most cannot model camera distortions • Capable to consider complicated imaging model with many parameters • Computationally expensive search procedure • Reasonable good initial guess for convergence of the solutions Technique: Linearity

  6. Major Drawback of existing algorithm • Calibrate with predefined pattern • Relating image projections to the camera parameters • Recent algorithms suffer from the same limitation • New discovery: Active Calibration

  7. Active Calibration • Camera capable of panning and tilitng can automatically calibrate itself • Modeled from eye movement • Active machines can keep track of object of interest • Facilitate region-of-interest process

  8. Active Calibration – How different? • Does not need a starting estimate for focal length and image center • Does not need prior information about focal length • Does not need to match points or feature b/w images • Reasonably accurate localization of contour • Estimate of center (Not too far from true value)

  9. Method of Calibration • Using perspective distortion to measure calibration parameters • Without using perspective distortion

  10. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  11. Theoretical Derivation

  12. Theoretical DerivationLemma 1 • Camera rotates by R and translate by T • New image contours

  13. Theoretical DerivationLemma 1 - Proof • Use two set of equation • ,

  14. Theoretical DerivationProposition 1 • Depth (Z) is larger than ΔX, ΔY, ΔZ • Camera moves by small tilt angle

  15. Theoretical DerivationProposition 1 – Proof • Rotation matrix R at small tilt angle • are negligible • From Lemma 1

  16. Theoretical DerivationProposition 1 – Proof • Expand right side of equation with Taylor series, because of small θt • With the same assumption, if camera moves by small pan angle θp

  17. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  18. Strategy for Active Calibration • Want – A relation b/w lens parameters and image information w/ given image contours before & after camera motion • Relate focal length to other camera parameters and the pan/tilt angles

  19. Strategy for Active CalibrationProposition 2 • Similar assumption as Proposition 1 • Center of the lens is estimated with a small error (δx, δy)

  20. Strategy for Active CalibrationProposition 2 – Proof • From Proposition 1 • Estimate image Center with error (δx, δy) • Ignore δxδy term

  21. Strategy for Active CalibrationProposition 3 (Plan A) • Using tilt (or pan) movement and considering three independent static contours, two linear equation in δxδy can be obtained if negligible terms are ignored

  22. Strategy for Active CalibrationProposition 3 – Proof • Two different contour, C1 and C2 • Point lying on C1 & C2, (x(1),y(1)) and (x(2),y(2)) • From Proposition 2

  23. Strategy for Active CalibrationProposition 3 – Proof • Equate right side equations and simplify where

  24. Strategy for Active CalibrationProposition 3 – Proof • Third contour, C3 • Point on C3, (x(3),y(3)) where

  25. Strategy for Active CalibrationProposition 3 – Proof • Finding fx and fy with estimated center • “e” denote the estimate of a certain parameter

  26. Strategy for Active Calibration Procedure Summary for Plan A • Estimate δx and δy using (3) and (4) with three distinct image contour • Obtain estimate for fx and fy by substituting resulting estimate into (5) and (6) • Term and make (5) and (6) unstable

  27. Strategy for Active Calibration Procedure Summary for Plan A • Variation in x-coord. for any point is due to change in perspective distortion (tilt) • Little change in the image y-coord. corresponding to a given 3-d point (pan) • and are small (few pixel) • Relative error can be large • presence of noise and inaccuracies in localization of a contour • Estimate in (5) & (6) are often unreliable

  28. Strategy for Active Calibration Proposition 4 (Plan B) • Using a single contour and pan/tilt camera movements fx and fy can be obtained if negligible terms are ignored

  29. Strategy for Active Calibration Proposition 4 – Proof • δx and δy are non-zero in the second equation in Proposition 1 • Simplify • The last three terms are negligible even if δx and δy are large

  30. Strategy for Active Calibration Proposition 4 – Proof • Simplifying eq (7) • fx can be obtained with similar way

  31. Strategy for Active Calibration Proposition 4 – Corollary • Given two independent contours, pan/tilt camera movements, and estimate of fx and fy given by and respectively, δx and δy can be obtained by solving • Considering from two independent contour from Proposition 2

  32. Strategy for Active Calibration Proposition 4 – Proof • Consider (8) • Most practical system • y < 300, fy > 500 • (8) is in form • A = 1, B < 0, C is small compare to B

  33. Strategy for Active Calibration Procedure Summary for Plan B • Estimate fx and fy from (12) and (13) using a single image contour • Solve for δx and δy by substituting resulting estimates into (10) and (11) and using another independent contour

  34. Strategy for Active Calibration Proposition 5 • When there is error in contour localization after pan/tilt movements, the ratio of the error in Plan A compared to Plan B for estimating fx(fy)is approximately

  35. Strategy for Active Calibration Proposition 5 – Proof • Introduce similar error term in and in (13) and (5) respectively • Simplify the expressions and consider the approximate magnitude of error in both the expressions • Take the ratio of these two terms

  36. Strategy for Active Calibration Proposition 5 – Implication • Error in Plan A can be as large as 30 times that of Plan B, for estimating focal lengths • Plan A is theoretically more precise, but not reliable for noisy real scenes

  37. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  38. Theoretical Error Analysis • Effect of errors from various sources on the estimation of different parameters • Errors in measurements of pan/tilt angles • Effect of noise in the extraction of image contours

  39. Theoretical Error Analysis • Remark 1 • Error in measurement of the pan (tilt) angle generates a proportional error in the estimate of fx(fy) • Proof • Consider(5) • fx is proportional to the pan angle • Any error in the measurement translates to a corresponding error in fx • Any error from tilt angle generate a proportional error in fy

  40. Theoretical Error Analysis • Remark 2 • Errors in measurement of the pan & tilt angles do not affect the estimate of the lens center • independent contours from the same image are considered • Proof • Linear equations in δx and δy are obtained by equating the right hand sides of two equations

  41. Theoretical Error Analysis • Consider (1) & (2) • Denote ε1: error in tilt angle • Contour extracted from same image • Then (θt+ε1) of (3) cancels out from both sides • Error in pan/tilt angle do not affect the estimate of lens center

  42. Theoretical Error Analysis • Consider two independent images generating the contours in (1) & (2) • K1 in (3) modifies to • is not equal to 1 in general • Errors in angle can change the estimate of the lens center if contours from independent images are considered

  43. Theoretical Error Analysis • Remark 3 • The coefficients of the linear (3)-(6) are unbiased in the presence of uncorrelated noise with zero mean • Coefficients involve a linear combination of terms • These terms are unbiased in the presence of uncorrelated noise with zero mean

  44. Theoretical Error Analysis • Remark 4 • The variance of the coefficients of (3)-(6) is inversely proportional to the number of points on a contour • Uncorrelated noise with zero mean is considered • Form of variances • Inversely proportional to the number of points for which the averages were computed

  45. Outline • Introduction • Theoretical Derivation • Strategies for Active Calibration • Theoretical Error Analysis • Experimental Result • Conclusion and Future Work

  46. Experimental ResultSimulation – Validity of Algorithms • Synthetic data used • Three independent contour represented by three sets of 3D points • Points projected onto the image plane • Values quantized to the nearest integer • Without noise, A produced more accurate estimate • Less than 1 percent relative error in focal length estimate

  47. Experimental ResultSimulation – Variation of error in focal length estimate • Change fx and fy from 100 to 1000 with interval 100 • Keep other parameters fixed • Discretization error influence A more when focal length was small • A is not very robust to noise • Larger the focal length, smaller the error relative to the focal length • Better estimate production with A

  48. Experimental ResultSimulation – Variation of error in focal length estimate • Error estimate from B does not drop off rapidly • B is theoretically less accurate than A

  49. Experimental ResultSimulation – Gaussian Noise Added • Poor performance with A • 20, 28, and 40 percent error with 3, 4, and 5 noise standard deviation • High robustness with B

  50. Experimental ResultTracking Contour • Match contours of interest during pan/tilt • For automatic calibration • Edges in the original image was thickened using the morphological operation of “dilation” • Edges after pan/tilt was AND-ed with the dilated image to extract corresponding contours after camera rotation

More Related