Learning more about nokiacv a mobile based computer vision algorithm suite l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 77

Learning More About NokiaCV A Mobile Based Computer Vision Algorithm Suite PowerPoint PPT Presentation

Learning More About NokiaCV A Mobile Based Computer Vision Algorithm Suite Current Scenario Mobile cameras are widespread Main uses for camera: capturing images/videos Still significant barriers to create new applications There is not enough support for camera/imaging in S60 SDK

Related searches for Learning More About NokiaCV A Mobile Based Computer Vision Algorithm Suite

Download Presentation

Learning More About NokiaCV A Mobile Based Computer Vision Algorithm Suite

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Learning more about nokiacv a mobile based computer vision algorithm suite l.jpg

Learning More About NokiaCVA Mobile Based Computer Vision Algorithm Suite


Current scenario l.jpg

Current Scenario

  • Mobile cameras are widespread

  • Main uses for camera: capturing images/videos

  • Still significant barriers to create new applications

  • There is not enough support for camera/imaging in S60 SDK

  • Only a handful of small companies making camera apps. Most of these are experts – computer vision expertise, image/signal processing experience


Solution nokia computer vision library nokiacv l.jpg

Solution: Nokia Computer Vision Library (NokiaCV)

  • Developers do not have to start imaging/camera applications from scratch

  • Shared platform will encourage further and more complex work in imaging domain

  • Nokia Computer Vision library built on top of Symbian

  • Fundamental operations for imaging and camera processing


Prior work l.jpg

Prior work

  • A few libraries available for mobile devices, including S60

    • Symbian mainly supports bitmap drawing, image/video capture.

    • No processing support

    • Difficult/slow to access individual pixels – common operation in processing

  • Other S60 imaging libraries are research-quality code, no support


Slide5 l.jpg

Goal

  • Internally to be used in imaging products, as a research test bed, etc.

  • Externally to provide developers in binary form to facilitate third-party imaging

  • Intel’s OpenCV is the desktop version of what NCV will become for mobiles

    • Full functionality

    • Wide range of imaging operations, however some not possible on mobile (yet)

    • NokiaCV’s main difference: supports operations that are useful and fast right now.


Use cases l.jpg

Use Cases


Capture enhancement l.jpg

Capture Enhancement

  • Image color conversions, imaging wrapper to modify images easily and manipulate pixels directly and easily

  • Building blocks present for developers to make high-quality stitching algorithms for panoramas

  • Building blocks present for developers to implement complex algorithms without requiring the building and debugging of a platform


Gaming ui l.jpg

Gaming/UI

  • Provides robust motion estimation as a software component, ready for games and applications


Post capture editing l.jpg

Post-capture editing

  • Image warping operation provided to create standard warping or in other entertainment uses

  • Image compositing provided to add items to captured images


Nokiacv library overview l.jpg

NokiaCV Library Overview

  • Library builds on OS, extending imaging capabilities

  • Image object provided standardizing OS image internals

  • Standard image operations provided

  • Linear algebra used in many advanced imaging applications, present in NCV

  • Building block for future advanced libraries, both externally and internally


Functional blocks l.jpg

Functional Blocks


Slide12 l.jpg

  • CCamus

  • CEgoMovement

  • CEigenvalues

  • CFixed

  • CImageOp

    • CArithmeticOp

    • CColorConversionOp

    • CComparisonOp

    • CConvolutionOp

    • CCornerDetectionOp

    • CEdgeDetectionOp

    • CGaussianSmoothOp

    • CMedianFilterOp

    • CMorphologicalOp

    • CResizeOp

    • CRotationOp

    • CShearOp

    • CTemplateMatchOp

    • CThresholdOp

    • CWarpOp

  • CImageOperations

  • CImagePyramid

  • CImageStatistics

  • CMotionHistoryImage

  • CNokiaCVImage

  • CNokiaCVMath

  • CNokiaCVMatrix

    • CNokiaCVVector

  • COpticalFlow

  • CPixelAccess

  • CPixelColorModel

  • CSVD

  • TColorModel

    • CHsvColorModel

    • CLabColorModel

    • CLmsColorModel

    • CRgbColorModel

    • CXyzColorModel

    • CYuvColorModel


Api overview l.jpg

API Overview

  • This library extends the image processing and math capabilities of the Symbian Series 60 platform, targeted for applications using computer vision techniques on camera phones.

  • The library provides an image class, CNokiaCVImage for handling RGB, grayscale and black and white images. Provides functions to perform arithmetic operations, transformations, feature extraction, color conversions. In addition, operations to compute image statistics such as histograms and pixel color means are provided.

  • The library has two algorithms (CCamus and CMotionHistoryImage) to determine motion estimates and history.

  • Matrix and vector classes CNokiaCVMatrix and CNokiaCVVector allow creation of matrices and vectors of arbitrary dimensionality. Math operations and some linear algebra techniques that work with these classes are available in the library.


Library features l.jpg

Library Features


Cnokiacvimage class l.jpg

CNokiaCVImage Class

  • CNokiaCVImage is a wrapper for CFbsBitmap that is provided by the platform. This class provides DisplayMode independent access to the pixel data.

    CFbsBitmap* iInternalBitmap;

  • Usage examples:

    // load from file

    CNokiaCVImage* incvBitmap = CNokiaCVImage::NewL();

    TRAPD(err1, incvBitmap->CreateL( *aImageFullName )); ...

    // draw bitmap

    CWindowGc& gc = SystemGc();

    gc.BitBlt( TPoint( 0,0 ), incvBitmap-> Bitmap() );


Functions l.jpg

Functions

  • static IMPORT_C CNokiaCVImage* CNokiaCVImage::NewLC ( ) ; Implements two-phase construction functions (NewLC(), ConstructL()) to create a new instance of the object.

    Returns: CNokiaCVImage*

  • IMPORT_C void CNokiaCVImage::CreateL (const TDesC & aFileName  ) ; Create image from a file.

  • static IMPORT_C void CNokiaCVImage::Copy (CNokiaCVImage * aSource, CNokiaCVImage * aTarget, TInt aBlendingRatio = 100 ); Copy a source image to the target image.


Cpixelaccess class l.jpg

CPixelAccess Class

  • CPixelAccess class provides access to CNokiaCVImage on a pixel-level. Derived from TBitmapUtil.

  • Example:

    CPixelAccess* out = CPixelAccess::NewL(aTarget);

    CPixelAccess* in = CPixelAccess::NewL(iImage, out);

    ...

    in->SetPos( TPoint( rotX, rotY ) );

    out->SetPos(TPoint( x, y ) );

    out->SetRGB( in->GetRGB() ); // set pixel value from in to out

    ...

    delete out;

    delete in;


Functions18 l.jpg

Functions

  • static IMPORT_C CPixelAccess* CPixelAccess::NewLC (CNokiaCVImage * aImage  ) ; Constructor. Sets the current pixel position to (0,0) position.

  • IMPORT_C TUint32 CPixelAccess::GetRGB ( ) ; Return pixel from the current position as a RGB.

  • IMPORT_C void CPixelAccess::SetRGB (TUint32 aPixel  ) ; Set pixel to the current position.

    Parameters:

    aPixel RGB value as TUint32 in format used by TRgb: 0x00BBGGRR


Cpixelcolormodel class l.jpg

CPixelColorModel Class

  • IMPORT_C TUint8 CPixelColorModel::Red ( ) ; Get the red component;

    Returns: TUint8 Red-component

  • IMPORT_C CYuvColorModel CPixelColorModel::ToYUV ( ) ; Convert to YUV.

    Returns: CYuvColorModel.

  • IMPORT_C CHsvColorModel CPixelColorModel::ToHSV ( ) ;

  • Convert to HSV

    return CHsvColorModel


Tcolormodel class l.jpg

TColorModel Class

  • class CRgbColorModel : public TColorModel

    {

    public:

    TInt iR;

    TInt iG;

    TInt iB;

    };

  • class CYuvColorModel : public TColorModel

    {

    public:

    TInt iY;

    TInt iU;

    TInt iV;

    };


Cnokiacvmath class l.jpg

CNokiaCVMath Class

  • This class gathers some utility functions in one place.

  • static IMPORT_C TInt SinLUT_Rad (TReal &aTrg, const TReal &aSrc) Computes Sin using lookup table.

  • static IMPORT_C TInt TanLUT_Rad (TReal &aTrg, const TReal &aSrc) Computes Tan using lookup table.


Cnokiacvmatrix class l.jpg

CNokiaCVMatrix Class

  • Fixed point matrix representation. Usage:

    CNokiaCVMatrix* m = CNokiaCVMatrix->NewLC(5,5);

    (*m)(3,3) = 2; (*m)(2,3) = 4.23;

    CNokiaCVMatrix* n = CNokiaCVMatrix->Ones(5,5); CleanupStack::PushL(n);

    CNokiaCVMatrix *res = (*m)*(*n)*5;

    CleanupStack::PushL(res);

    CFixed det = res->Det();

    ...

    CleanupStack::Pop(m); CleanupStack::Pop(n); CleanupStack::Pop(res);

    delete m; delete n; delete res;


Functions23 l.jpg

Functions

  • static IMPORT_C CNokiaCVMatrix * NewL (const TInt aRows, const TInt aCols) Creates a new matrix according to size given with initial value zero.

  • IMPORT_C CNokiaCVMatrix * Inverse () const Calculates the inverse matrix of this matrix.

  • IMPORT_C CFixed ** Eigenvalues () Calculates the eigenvalues of this matrix.

  • IMPORT_C CFixed * Singularvalues () Calculates the singular values of this matrix using ncvSVD.


Csvd class l.jpg

CSVD Class

  • Singular value decomposition of a given real matrix.

  • Householder bidiagonalization and a variant of the QR algorithm are used.

  • The SVD algorithm is derived from JAMA/TNT, the Java Matrix Package.

  • Usage:

    CNokiaCVMatrix *a = CNokiaCVMatrix::NewL(rows,cols);

    CSVD *svd;

    TRAPD(err, svd = CSVD::NewL(*a))

    ... //Handle possible errors

    CNokiaCVMatrix *v = svd->V();

    ...

    delete a; delete svd; delete v;


Cnokiacvvector class l.jpg

CNokiaCVVector Class

  • Fixed point representation of a vector.

  • class CNokiaCVVector : public CNokiaCVMatrix

    { …

  • IMPORT_C TInt Length () Getter of the vector size/length.

  • IMPORT_C CNokiaCVVector * Normalize () Creates the normalization of this vector.

  • IMPORT_C CNokiaCVVector * CrossProduct (CNokiaCVVector &aVector) Calculates the cross product of this and the given vector.

  • IMPORT_C CFixedDotProduct (CNokiaCVVector &aVector) Calculates the dot product of to same size vectors.


Cmotionhistoryimage class l.jpg

CMotionHistoryImage Class

  • This class computes a motion history image (MHI) for a series of input frames. The MHI is a grayscale bitmap in which recently moved pixels are represent with bright values and "older" with darker values. Frame-by-frame the values from previous frames are decayed linearly.


Functions27 l.jpg

Functions

  • static IMPORT_C CMotionHistoryImage * NewL (CNokiaCVImage &aImage, const TInt aHistorySize) Construct CMotionHistoryImage from the first image of motion history.

    Parameters:

    aImageSize Size of the images. aHistorySize Maximum number of images in the history.

  • IMPORT_C void AddImage (CNokiaCVImage &aImage) Add an image to the motion history.

  • IMPORT_C void GetImage (CNokiaCVImage &aImage) Get a CNokiaCVImage containing the motion history image.


Copticalflow class l.jpg

COpticalFlow Class

  • The optical flow class: motion estimations for each pixel. This class also implements methods for calculating higher level information from the optical flow.

  • Friends class CCamus: Camus algorithm for estimation of optical flow from a series of bitmaps.

  • Member CEgoMovement Class: class for pixel motion estimation values.

    CFixediX Movement along the x-axis. CFixediY Movement along the y-axis.


Functions29 l.jpg

Functions

  • static IMPORT_C COpticalFlow * NewL (const TSize &aImageSize) Factory method for creating an instance. Parameters:

    • aImageSize Size of the optical flow map

  • IMPORT_C CEgoMovementMean () const Calculate the mean value of the optical flow. Represents tilting and panning of the camera.

  • IMPORT_C CFixedRotation () Calculate the mean rotation of the optical flow in the image plane.


Functions30 l.jpg

Functions

  • void SetCamusOpts (const TInt aCamusOpts) Set runtime parameters of the Camus motion estimation algorithm.

  • IMPORT_C CEgoMovement (const CFixed aX, const CFixed aY) Constructor.

  • IMPORT_C CFixedNorm () Calculates the euclidian norm of the movement.


Cimagepyramid class l.jpg

CImagePyramid Class

  • Creates a Laplacian or Gaussian image pyramid from a given bitmap.

  • IMPORT_C CNokiaCVImage * LevelImage (TInt aLevel) Getter for an image of certain level.

    Returns:

    • CNokiaCVImage* Pointer to the level image

  • static IMPORT_C CImagePyramid * NewLC (CNokiaCVImage *aImage, TInt aLevels, TImagePyramidType aType=EGaussianPyramid) Constructor.

    Parameters:

    • aImage CNokiaCVImage to construct the pyramid from aLevels How many levels in the pyramid. If the number is too large it will be truncated. aType Which type of pyramid to construct. Default = Gaussian.


Cimagestatistics class l.jpg

CImageStatistics Class

  • Calculating various statistics from CNokiaCVImages. CImageStatistics provides the following statistical functions for CNokiaCVImages:

    Histogram

    Mean

    Standard deviation

    Image moments:

    • m00

    • m01

    • m10

    • m11

    • m20

    • M02

  • Usage:

    TReal mean; CImageStatistics::Mean(*iSourceBitmap, mean, (TChannel)aChannel);


Functions33 l.jpg

Functions

  • static IMPORT_C void Histogram (CNokiaCVImage &aImage, THistogram &aArray, TChannel aChannel=EGrayChannel) Calculate the histogram from a CNokiaCVImage.

  • static IMPORT_C void Mean (CNokiaCVImage &aImage, TReal &aMean, TChannel aChannel=EAllChannels) Calculate the mean of CNokiaCVImage. static IMPORT_C void StDev (CNokiaCVImage &aImage, TReal &aStDev, TReal &aMean, TChannel aChannel) Calculate the standard deviation of a CNokiaCVImage.


Cimageoperations class l.jpg

CImageOperations Class

  • Provides an simplified access to image operations. Each operation is a static method that creates and returns the result bitmap.

  • Usage

    CNokiaCVImage* iTarget = CImageOperations::Rotate(iSource, 45);

    ...

    delete iTarget;


Functions35 l.jpg

Functions

  • static IMPORT_C CNokiaCVImage * Resize (CNokiaCVImage *aSource, TSize aSize, TBool aBilinear=EFalse) Resizes the source image. See CResizeOp for more information.

  • static IMPORT_C CNokiaCVImage * Convolution (CNokiaCVImage *aSource, CNokiaCVMatrix &aKernel, TInt aNormFactor) Performs convolution of CNokiaCVImage and a kernel (CNokiaCVMatrix).

  • static IMPORT_C CNokiaCVImage * Gaussian (CNokiaCVImage *aSource, TUint aSize) Gaussian smoothing filter.

  • static IMPORT_C CNokiaCVImage * EdgeDetectLaplacian (CNokiaCVImage *aSource) Laplacian edge detection filter. See CEdgeDetectionOp for more info.


Ccamus class l.jpg

CCamus Class

  • Implements the Camus algorithm for estimation of optical flow from a series of bitmaps. This class creates an COpticalFlow object that contains an estimation of the motion vector (x and y motion in pixels) for each pixel of the input frame.

  • Can use the methods of COpticalFlow to extract higher level information such as estimated camera rotation/movement.

  • The algorithm is highly parameterizable.

  • Features of the algorithm and some hints:

    • Maximum estimated motion is 1 or 2 pixels/frame, so it is typically reasonable to use small bitmaps (e.g. 16x12 pixels).

    • History of several frames is used to achieve sub pixel accuracy

    • Based on template matching

    • Uses grayscale images

    • Large motions are hard so having a good frame rate is essential since the per-frame motion is inversely proportional to the frame rate

    • Small bitmaps are preferred (especially when estimation of quick movements is required) as its faster to compute; higher frame rate.


Camus algorithm l.jpg

Camus Algorithm

Basic idea:

  • Simple technique to perform optical flow computations using correlation or patch-making methods. The idea here is to match features between images and computing the SD (sum-difference) of patches.

  • Reference: T. Camus, Real-Time Optical Flow, PhD Thesis, Brown University Technical Report CS-94-36, 1994.


Functions38 l.jpg

Functions

static IMPORT_C CCamus* CCamus::NewL

(CNokiaCVImage & aImage,

const TInt aHistorySize = KDefaultCamusHistorySize )

IMPORT_C void CCamus::GetOpticalFlow (COpticalFlow & aOpticalFlow, CNokiaCVImage & aImage, const TInt aCamusOpts = KDefaultCamusOpts )


How to use ccamus l.jpg

How to Use CCamus

COpticalFlow* iOpticalFlow = COpticalFlow::NewL(iFirstFrame->Size());

CNokiaCVImage* iCamus = CCamus::NewL(*iFirstFrame, 5);

iCamus->GetOpticalFlow(*iOpticalFlow, *iSecondFrame, CAMUS_SETCAMUSOPTS(CCamus::EMedium, CCamus::EMedium, 3));

CEgoMovement mean = iOpticalFlow->Mean();


Cegomovement class l.jpg

CEgoMovement Class

  • Simple vector-like wrapper class for pixel motion estimation values.


Functions41 l.jpg

Functions

  • IMPORT_C CEgoMovement::CEgoMovement (const CFixedaX, const CFixedaY )Constructor.

    Parameters:

    • aX Movement along the x-axis.

    • aY Movement along the y-axis.

  • IMPORT_C CFixed CEgoMovement::Norm ( ) Calculates the Euclidian norm of the movement.

    Returns:

    • Norm (the length of the movement in cartesian coordinate system).

  • IMPORT_C CFixed CEgoMovement::DotProduct (const CEgoMovement & aMovement  ) Calculates a dot product.

    Parameters:

    • aMovement Vector movement.

      Returns:

    • Dot product value.


Ceigenvalues class l.jpg

CEigenValues Class

  • Eigenvalue decomposition of a real matrix. Usage:

    CNokiaCVMatrix * matrix = ...; CEigenvalues * eigs;

    TRAPD(err, eigs = CEigenvalues->NewL(matrix));

    if(err != KErrNone) {

    //eigs not found do something

    }

    else {

    TReal * real = eigs->RealEigenvalues();

    delete eigs;

    } ...


Functions43 l.jpg

Functions

  • IMPORT_C TReal* CEigenvalues::RealEigenvalues ( )

    Getter for the real part of the eigenvalues (iD).

    Returns:

    • the real part of the eigenvalues in an array

  • IMPORT_C TReal* CEigenvalues::ImaginaryEigenvalues ( )

    Getter for the imaginary part of the eigenvalues (iE).

    Returns:

    • the imaginary part of the eigenvalues in an array


Cfixed class l.jpg

CFixed Class

  • Fixed point class.

  • The class implements basic arithmetic for fixed point numbers.

  • CFixed can be used instead of TReal when performance more critical than accuracy.


Cimageop class l.jpg

CImageOp Class

  • Image operations interface.

  • This class represents an interface that all image operation classes must implement. Most of the operations implementing this interface are also included in CImageOperations.

  • There are various image operations that implement this interface, e.g. for rotating, scaling and filtering bitmap.

    Usage example (CRotationOp):

    iSource = ....

    iTargetMode = EGray256;

    iTarget1 = CNokiaCVImage::NewL();

    iTarget1->CreateL(iCurrSource->Size(), iTargetMode);

    CRotationOp op(iCurrSource, 90); op.DoOperation(iTarget1); ...


Functions46 l.jpg

Functions

  • virtual IMPORT_C void CImageOp::DoOperation (CNokiaCVImage * aTarget  ) [pure virtual]

  • All image operations must implement this method. This method does the actual operation.

  • Typically you need to allocate and create the target bitmap yourself before doing the operation

  • Parameters:

    • aTarget target bitmap

  • Note:

    • the function can leave

  • Implemented in CArithmeticOp, CColorConversionOp, CComparisonOp, CConvolutionOp, CCornerDetectionOp, CEdgeDetectionOp, CGaussianSmoothOp, CMedianFilterOp, CMorphologicalOp, CResizeOp, CRotationOp, CShearOp, CTemplateMatchOp, CThresholdOp, and CWarpOp.


Carithmeticop l.jpg

CArithmeticOp

  • CArithmeticOp provides basic arithmetic operations for CNokiaCVImages.

  • Target and source bitmaps must be of same size.

  • Usage:

    IMPORT_C void CArithmeticOp::DoOperation (CNokiaCVImage * aTarget  ) 

    Does the actual operation.

    Parameters:

    • aTargetCNokiaCVImage to store the result in. Must be allocated and created beforehand.

      Implements CImageOp.


Image arithmetic examples l.jpg

Image Arithmetic Examples


Ccolorconversionop class l.jpg

CColorConversionOp Class

  • Provides CNokiaCVImage with color conversions.


Functions50 l.jpg

Functions

  • IMPORT_C CColorConversionOp::CColorConversionOp (CNokiaCVImage * aImage, TDisplayMode aTargetMode )

  • Constructor. Converts given image to given display mode.

  • Parameters:

    • aImage Image to convert aTargetMode Target display mode Valid TDisplayModes for aTargetMode are:

      • EGray2

      • EGray4

      • EGray16

      • EGray256

      • EColor4K

      • EColor64K

      • EColor16M

  • IMPORT_C void CColorConversionOp::DoOperation (CNokiaCVImage * aTarget  ) 

  • Parameters:

    • aTargetCNokiaCVImage to store the result in.


  • Ccomparisonop class l.jpg

    CComparisonOp Class

    • Provides CNokiaCVImage with pixel-wise image comparison operations: "greater than", "less than" and "equal".

    • Matching pixels are set white. Non-matching and pixels outside the comparison region are set black.

    • Valid values are:

      • EGreater

      • ELess

      • EEqual


    Functions52 l.jpg

    Functions

    • IMPORT_C CComparisonOp::CComparisonOp (CNokiaCVImage * aImage1, CNokiaCVImage * aImage2, TComparisonTypeaComparison )Constructor.

      Parameters:

      • aImage1 images to compare aImage2 another images to compare. aComparison Comparison operation: .EGreater, ELess or EEqual

    • IMPORT_C CComparisonOp::CComparisonOp (CNokiaCVImage * aImage1, CNokiaCVImage * aImage2, TComparisonTypeaComparison, TRect aSubregion )Constructor.

      Parameters:

      • aImage1 images to compare aImage2 another images to compare. These two images must be of same bit depth and same size

      • aComparison Comparison operation: .EGreater, ELess or EEqual

      • aSubregion subregion of images 1 and 2 that are compared

    • IMPORT_C void CComparisonOp::DoOperation (CNokiaCVImage * aTarget  )

      Parameters:

      • aTargetCNokiaCVImage to store the result in. The target bitmap must be black and white.


    Cconvolutionop class l.jpg

    CConvolutionOp Class

    • Convolution operation.

    • Performs convolution of CNokiaCVImage and a kernel (CNokiaCVMatrix).

    O57 = I56K11 + I58K12 + I59K13 + I67K21 + I68K22 + I69K23


    Functions54 l.jpg

    Functions

    • IMPORT_C CConvolutionOp::CConvolutionOp (CNokiaCVImage * aImage, CNokiaCVMatrix & aKernel ) Constructor.

      Parameters:

      • aImageCNokiaCVImage to use as source aKernelCNokiaCVMatrix Kernel to use for convolution. Only 3x3 and 5x5 matrices are supported.

    • IMPORT_C void CConvolutionOp::DoOperation (CNokiaCVImage * aTarget  ) 

      Parameters:

      • aTargetCNokiaCVImage to store the result in.


    Ccornerdetectionop l.jpg

    CCornerDetectionOp

    • Corner detection for CNokiaCVImage.

    • Moravec corner detector is implemented.

      • Defines interest points as points where there is a large intensity variation in every direction i.e. corners.


    Functions56 l.jpg

    Functions

    • IMPORT_C CCornerDetectionOp::CCornerDetectionOp (CNokiaCVImage * aImage, TCornerDetectionMethodaMethod = EMoravec, TInt aNeighborhoodSize = 20, TInt aConstant = 25 ) Constructor.

      Parameters:

      • aImageCNokiaCVImage to detect from

      • aMethod Which corner detection method to use

      • aNeighborhoodSize Size of the neighborhood to use for adaptive thresholding

      • aConstant a constant value to subtract from the mean threshold

    • IMPORT_C void CCornerDetectionOp::DoOperation (CNokiaCVImage * aTarget  ) 

      Parameters:

      • aTargetCNokiaCVImage to store the result


    Cedgedetectionop l.jpg

    CEdgeDetectionOp

    • Edge detection filters using convolution.

    • Possible values are:

      • ESobel

      • EPrewitt

      • ELaplacian

      • ECanny

    Canny edge detection with different convolution sizes


    Functions58 l.jpg

    Functions

    • IMPORT_C CEdgeDetectionOp::CEdgeDetectionOp (CNokiaCVImage * aImage, TEdgeDetectionMethodaMethod ) Constructor.

      Parameters:

      • aImageCNokiaCVImage to detect from aMethod Which edge detection method to use

    • IMPORT_C void CEdgeDetectionOp::DoOperation (CNokiaCVImage * aTarget  ) 

      Parameters:

      • aTargetCNokiaCVImage to store the result


    Cgaussiansmoothop l.jpg

    CGaussianSmoothOp

    • Gaussian smoothing operator.

    • The Gaussian smoothing operator is a 2-D convolution operator used to `blur' images and remove detail and noise.

    • The shape of the kernel represents the shape of a Gaussian (`bell-shaped') point-spread function.

    5 X 5 Gaussian Kernel


    Functions60 l.jpg

    Functions

    • IMPORT_C CGaussianSmoothOp::CGaussianSmoothOp (CNokiaCVImage * aImage, TInt aKernelSize ) Constructor.

      Parameters:

      • aImageCNokiaCVImage a source image aKernelSize The number of rows and columns in Gaussian kernel: Possible values are. 3 and 5.

    • IMPORT_C void CGaussianSmoothOp::DoOperation (CNokiaCVImage * aTarget  ) 

      Parameters:

      • aTargetCNokiaCVImage to store the result


    Cmedianfilterop l.jpg

    CMedianFilterOp

    • Median filter operation.

    • The median filter is normally used to reduce noise in an image.

    • Does a reasonable job of preserving useful detail in the image.

    • Replaces every pixel with the medianof its neighboring pixels.

    Median Filter 3X3


    Functions62 l.jpg

    Functions

    • IMPORT_C CMedianFilterOp::CMedianFilterOp (CNokiaCVImage * aImage, TInt aKernelRows ) Constructor.

    • Parameters:

      • aImageCNokiaCVImage to use as source aKernelRows How big a kernel to use. Possible values are:

        • 3 for 3x3

        • 5 for 5x5

        • 7 for 7x7

  • IMPORT_C void CMedianFilterOp::DoOperation (CNokiaCVImage * aTarget  )

    Parameters:

    • aTargetCNokiaCVImage to store the result in.


  • Cmorphologicalop class l.jpg

    CMorphologicalOp Class

    • Dilation/Erosion filter.

    • Here, the state of any given pixel in the output image is determined by applying a rule to the corresponding pixel and its neighbors in the input image. The rule used to process the pixels defines the operation as a dilation or an erosion.

    • Dilation: The value of the output pixel is the maximumvalue of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1.

    • Erosion: The value of the output pixel is the minimumvalue of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0.


    Functions64 l.jpg

    Functions

    • IMPORT_C CMorphologicalOp::CMorphologicalOp (CNokiaCVImage * aImage, CNokiaCVMatrix & aStruct, TMorphologicalOperationaOperation ) Constructor.

      Parameters:

      • aImageCNokiaCVImage to use as source

      • aStructCNokiaCVMatrix a structuring element to use for morphing

      • aOperation Which operation to do

    • IMPORT_C void CMorphologicalOp::DoOperation (CNokiaCVImage * aTarget  )

      Parameters:

      • aTargetCNokiaCVImage to store the result in.


    Cresizeop class l.jpg

    CResizeOp Class

    • Resizes the CNokiaCVImage.

    • IMPORT_C CResizeOp (CNokiaCVImage *aImage, TBool aBilinear=EFalse) Constructor.

    • IMPORT_C void DoOperation (CNokiaCVImage *aTarget)

      Do the actual resize. Take the new size from target image and store the result in it.


    Crotationop class l.jpg

    CRotationOp Class

    • Rotates a CNokiaCVImage.

    • Implements an arbitrary angle rotation.

    • IMPORT_C CRotationOp (CNokiaCVImage *aImage, TInt aAngle) Constructor.

    • IMPORT_C void DoOperation (CNokiaCVImage *aTarget)

       Do the actual operation.


    Cshearop class l.jpg

    CShearOp Class

    • Shear operation for CNokiaCVImage.

    • Shears bitmap Horizontally or Vertically (creates a parallelogram).

    • IMPORT_C CShearOp (CNokiaCVImage *aImage, TInt aAngle, TDirection aDirection=EHorizontal) Constructor.

    • IMPORT_C void DoOperation (CNokiaCVImage *aTarget) 

      Do the actual operation.


    Ctemplatematch class l.jpg

    CTemplateMatch Class

    • Template matching operation for two CNokiaCVImages.

    • Used for finding small parts of an image that match a template image.

    • Algorithm:

      • This method is implemented by first creating a subimage (the template).

      • The center of the subimage is simply moved over each (x,y) point in the a candidate image.

      • The sum of products between the coefficients in the candidate image and the corresponding neighborhood pixels in the area spanned by the template is calculated to determine best match.

    • Also applicable for color images.


    Functions69 l.jpg

    Functions

    • IMPORT_C CTemplateMatchOp::CTemplateMatchOp (CNokiaCVImage * aImage, CNokiaCVImage * aTemplate )

      Constructor for template matching operation.

    • Parameters:

      • aImageCNokiaCVImage to match against aPattern

      • CNokiaCVImage containing the template to match

    • IMPORT_C void CTemplateMatchOp::DoOperation (CNokiaCVImage * aTarget = NULL  )

      Parameters:

      • aTargetCNokiaCVImage to store the resulting correlation values

        Implements CImageOp.

    • IMPORT_C void CTemplateMatchOp::DoOperation (TPoint & aPoint, CNokiaCVMatrix * aCorrMatrix = NULL )Parameters:

      • aPoint resulting matching poin

      • aCorrMatrix correlation matrix


    Cthresholdop l.jpg

    CThresholdOp

    • Provides CNokiaCVImage with a threshold operation.

    • Thresholding is the most common method of segmenting images into particle regions and background regions.

    • Creates an black and white bitmap from CNokiaCVImage by thresholding.

    • Implements two thresholding methods:

      • simple with static threshold value

      • Local adaptive: Selects an individual threshold for each pixel based on the range of intensity values in its local neighborhood. This allows for thresholding of an image whose global intensity histogram doesn't contain distinctive peaks.


    Functions71 l.jpg

    Functions

    • IMPORT_C CThresholdOp::CThresholdOp (CNokiaCVImage * aImage, TUint8 aThresholdValue )

      Constructor for thresholding with given value.

    • Parameters:

      • aImageCNokiaCVImage to apply the thresholding to.

      • aThresholdValue an integer value (0-255) to use as a threshold.

    • IMPORT_C CThresholdOp::CThresholdOp (CNokiaCVImage * aImage, TInt aSize, TInt aConstant, TThresholdType = EMean )

      Constructor for adaptive thresholding.

      Parameters:

      • aImageCNokiaCVImage to apply the thresholding to.

      • aSize aSize x aSize neighborhood to use for adaptation

      • aConstant Constant value that is subtracted from the mean.

    • IMPORT_C void CThresholdOp::DoOperation (CNokiaCVImage * aTarget  )

      Parameters:

      • aTargetCNokiaCVImage to store the result in.


    Cwarpop class l.jpg

    CWarpOp Class

    • Provides CNokiaCVImage with a warp operation The result of the operation is like dragging an elastic image from one point to a given direction.

    • IMPORT_C CWarpOp::CWarpOp (CNokiaCVImage * aImage, TPoint aSource, TPoint aDestination )

      Constructor.

      Parameters:

      • aImage Source image. aSource Source point to start the warp from. aDestination Destination point to warp towards.

    • IMPORT_C void CWarpOp::DoOperation (CNokiaCVImage * aTarget  ) 

      Parameters:

      • aTargetCNokiaCVImage to store the result in.


    Demos l.jpg

    Demos


    Outline l.jpg

    Outline

    • Cube

    • Motion Estimation

    • Image Processing

    • Game


    Slide75 l.jpg

    Cube

    • Using a cube to represent camera movement

      • Obtaining optical flow using Camus algorithm

      • Mean movement of optical flow  Cube’s movement along x and y

      • Mean rotation of optical flow  Cube’s rotation

      • Depth info: movement along z axis  Cube’s size

    • NokiaCV classes used

      • CCamus

      • CEgoMovement

      • COpticalFlow

      • CNokiaCVImage

      • CPixelAccess

      • CFixed


    Motion estimation l.jpg

    Motion Estimation

    • Calculate and display motion history image

    • Calculate optical flow, indicate horizontal and vertical motion

    • NokiaCV classes used

      • CMotionHistoryImage

      • CNokiaCVImage

      • CCamus

      • COpticalFlow


    Image processing l.jpg

    Image Processing

    • Loads two images

    • Perform image processing based on user selection

    • NokiaCV classes used

      • All image processing classes


  • Login