Environmental awareness
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

ENVIRONMENTAL AWARENESS PowerPoint PPT Presentation


  • 47 Views
  • Uploaded on
  • Presentation posted in: General

ENVIRONMENTAL AWARENESS. Mahaneeya Raman . Automated Activity and Object Detection from Soldier-worn sensors. To monitor and assist soldiers Monitor environment and soldier entity Data acquisition through: Omni-vision Camera Body sensors GPS/GIS. Agenda.

Download Presentation

ENVIRONMENTAL AWARENESS

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Environmental awareness

ENVIRONMENTAL AWARENESS

Mahaneeya Raman

Mahaneeya Raman - Extended Class Project


Automated activity and object detection from soldier worn sensors

Automated Activity and Object Detection from Soldier-worn sensors

  • To monitor and assist soldiers

  • Monitor environment and soldier entity

  • Data acquisition through:

    • Omni-vision Camera

    • Body sensors

    • GPS/GIS

Mahaneeya Raman - Extended Class Project


Agenda

Agenda

  • Brief overview of Omni-vision Camera System

  • Brief overview of body sensors

  • Overview of GPS/GIS

  • Image Stitching –procedure and problems encountered

  • BodyMedia SenseWear sensor system

Mahaneeya Raman - Extended Class Project


Omni vision camera system

Omni-vision Camera System

  • Wide angle lenses (f= 1.7 mm ~ 110 degrees)

  • Higher image resolution than mirror cameras and cheaper ones.

  • Cameras daisy chained.

  • Just 1 Fire wire cable (signal +power)

Mahaneeya Raman - Extended Class Project


Body sensors to predict physical activity and emotion

Body sensors – to predict physical activity and emotion

  • Physiological data related to human activity and emotion

  • Physical activity – accelerometer signals, temperature, heat flux

  • Human emotion – Skin conductance (Galvanic Skin Response), heart rate

Mahaneeya Raman - Extended Class Project


Gps gis

GPS/GIS

  • Map data to physical coordinates

  • Simplify the classification problem – index into previously collected databases

  • For visualization purposes – need Geographic Information System (GIS) data

  • Setback – extensive GIS data is not free!!

Mahaneeya Raman - Extended Class Project


Image stitching steps involved

Image Stitching – steps involved

  • Camera Calibration

  • Correction for radial and tangential distortion

  • Projection onto cylindrical coordinates

  • Image stitching

  • Take-away from this work:

    • Problems faced - Parallax error

    • Suggestion on how to solve

Mahaneeya Raman - Extended Class Project


Camera calibration

Camera Calibration

  • To correct radial distortion and tangential distortion in images

  • Radial – Straight lines in real world appear curved in the image plane

  • Tangential – Image not located on a strict plane surface

Mahaneeya Raman - Extended Class Project


Undistorting image and projection onto cylindrical coordinates

Undistorting image and projection onto cylindrical coordinates

  • Polynomial distortion model

  • For aligning the images

  • To fit together as a panorama

  • Conversion from Cartesian to Cylindrical coordinates:

    r2 = x2 + y2

    tan θ = y / x

Mahaneeya Raman - Extended Class Project


Images

Images

Distorted ImageCorrected image and

projected onto

cylindrical coordinates

Mahaneeya Raman - Extended Class Project


Image stitching

Image Stitching

  • Load the two undistorted images

  • Select points using

    cpselect function in Matlab. Frame1_points and Frame2_points saved.

  • Determine transformation

    T, between the 2 images,

    x’ = Tx

  • Map one image onto the other based on the transformation.

Mahaneeya Raman - Extended Class Project


Image stitching output

Image Stitching output

Final output

Mahaneeya Raman - Extended Class Project


Image stitching of raw images parallax error

Image stitching of raw images – Parallax error

Mahaneeya Raman - Extended Class Project


Understanding parallax error

Understanding Parallax error

  • Distance related measurement error

  • High for images that are close to the camera lens

  • Occurs because Same points are located at different distances from two camera lenses, in both images

  • Distances between 2 given points is not the same in both images

  • Solution :

    • non-linear mapping has to be done

    • Ex. Radial Basis Function method / full planar perspective models

Mahaneeya Raman - Extended Class Project


Bodymedia sensewear pro 2 armband

BodyMedia® SenseWear PRO2 Armband

  • Body sensor worn on upper right arm

  • Timestamp, Memory, Battery indicators

  • Connects to PC through USB cable

  • Collects through 6 continuous streams of data channels

  • Stores 30 channels of data

Mahaneeya Raman - Extended Class Project


Bodymedia sensewear pro 2 armband1

BodyMedia® SenseWear PRO2 Armband

  • 2 axis accelerometer

  • Heat flux sensor

  • Galvanic Skin Response sensor

  • Skin Temperature sensor

  • Near-Body Temperature Sensor

Mahaneeya Raman - Extended Class Project


Sensor signal graph for typing writing

Sensor signal graph for typing-writing

Skin Temperature

Longitudinal Accelerometer

Heat Flux

Transverse Accelerometer

Galvanic Skin Response

Mahaneeya Raman - Extended Class Project


Environmental awareness

Applications of SenseWear

Mahaneeya Raman - Extended Class Project


Data collection

Data Collection

  • Pattern of data collected:

    • Typing – writing

    • Walking – simply sitting

    • Playing ‘quake’ – watching a comedy clip

  • Sample rate - 4 samples/second

  • Video surveillance using fire-i cameras while data is being collected

  • Timestamp between activities

Mahaneeya Raman - Extended Class Project


Recorded video using 2 fire i cameras

Recorded video using 2 fire-i cameras

Mahaneeya Raman - Extended Class Project


Sensor signal graph for typing writing walking sitting playing quake relaxing

Sensor signal graph for typing-writing, walking - sitting, playing ‘quake’-relaxing

Longitudinal Accelerometer

Transverse Accelerometer

Heat Flux

Mahaneeya Raman - Extended Class Project


Data analysis

Data Analysis

  • Generate signal graph and excel sheets using bodymedia’s innerview software

  • Notice Accelerometer signals change considerably

  • Use a classifier algorithm

  • 50% data – training

  • 50% data – testing

Mahaneeya Raman - Extended Class Project


Knn and lda classifier considered algorithms for comparison

KNN and LDA classifier – considered Algorithms for comparison

  • KNN – K Nearest Neighbor Algorithm

    • If x – to be classified, and (x1, y1), . . . , (xk, yk) are x’s k nearest neighbors, and d(x, xi) = distance between x and xi, x is classified into the nearest neighbor cluster.

  • LDA

    • Method to find linear discriminant boundaries between K classes

    • Define K linear discriminant functions for K classes

    • Classify x to the class with the largest value for its discriminant function

Mahaneeya Raman - Extended Class Project


Conclusion

Conclusion

  • Useful GIS data can be integrated with GPS data for effective localization and environment analysis

  • Parallax error – can be solved by applying a non-linear transformation like RBF or full planar perspective models

  • Pending - Compare classifier algorithms to classify activity, which can be extended to predict human emotion as well

Mahaneeya Raman - Extended Class Project


References

References

[1]CAMEO: The Camera Assisted Meeting Event Observer – Paul E. Rybski, Fernando de la Torre, Raju Patil, Carlos Vallespi, Manuela Veloso, Brett Browning.

[2]Image Warping - Mikkel B. Stegmann , Informatics and Mathematical Modelling, Technical University of Denmark.

[3]Creating Full View Panoramic Image Mosaics and Environment Maps - Richard Szeliski and Heung-Yeung Shum, Microsoft Research.

[4]16 papers on bodymedia applications - http://www.bodymedia.com/research/whitepapers.jsp

Mahaneeya Raman - Extended Class Project


Thank you

Thank you

Mahaneeya Raman - Extended Class Project


Matlab toolbox for calibration

Matlab toolbox for calibration

  • Intrinsic calibration – focal length for each image axis, an image center, 3 terms of radial distortion, and 2 terms of tangential distortion.

  • Checker-board method – straight lines with easily localizable end points and interior points can be found in several orientations throughout the image plane.

Mahaneeya Raman - Extended Class Project


Matlab toolbox for calibration1

Matlab toolbox for calibration

  • Images of checkerboard at different inclinations (horizontal, vertical, diagonal)

  • Provide size of the square

  • Extract grid corners of all images, one by one.

  • Provide size of window of squares chosen

  • Corner extraction - verification

Mahaneeya Raman - Extended Class Project


Distortion model plumb bob model

Distortion Model – “Plumb Bob” model

  • P – point in space coordinate vector,

    XXc= [Xc;Yc;Zc] in camera ref. frame

  • Project P on the image plane according to intrinsic parameters (fc, cc, alpha_c, kc)

  • xn – normalized (pinhole) image projection,

    xn =[ Xc/ Zc;Yc/ Zc] = [x ; y]

  • Let r2 = x2 + y2

Mahaneeya Raman - Extended Class Project


Distortion model plumb bob model1

Distortion Model – “Plumb Bob” model

  • After lens distortion, the new normalized point coordinate, xd = [xd(1) ; xd(2)]

    xd = (1+ kc(1)r2 + kc(2)r4 + kc(5)r6 )xn + dx

  • Where, dx – tangential distortion vector

    dx = [2kc(3)xy+ kc(4)(r2 + 2x2) ;

    kc(3)(r2 + 2y2)+ 2kc(4)xy ]

Mahaneeya Raman - Extended Class Project


Distortion model

Distortion Model

  • The final pixel coordinates, x_pixel = [xp;yp]

  • Xp = fc(1)(Xd (1) + alpha_c*Xd (2)) +cc(1)

  • Yp = fc(2) Xd (2) +cc(2)

  • Therefore, [xp;yp;1] = KK [xd(1);xd(2);1]

    where, KK = [fc(1)alpha_c*fc(1)cc(1) ;

    0fc(1)cc(1) ;

    001]

Mahaneeya Raman - Extended Class Project


Image stitching1

Image Stitching

  • Load 2 images, distortion corrected and projected onto cylindrical coordinates.

  • Select points using cpselect function in Matlab. Input_points and base_points saved.

  • Determine transformation T, between the 2 images,

    x’ = Tx

    x s cosα s sinα tx x

    y =-s sinα s cosα tyy

    1001 1

Mahaneeya Raman - Extended Class Project


Image stitching continued

Image Stitching - continued

  • By rearranging the equation so the warping parameters is the vector t in,

    x’ = Zt

    x’xy100s cosα

    y’ =y-x010s sinα

    1000 01 tx

    ty

    1

Mahaneeya Raman - Extended Class Project


Radial basis function method

Radial Basis Function method

  • The input layer is the set of source nodes (sensory units).

  • The second layer is a hidden layer of high dimension.

  • The output layer gives the response of the network to the activation patterns applied to the input layer.

  • The transformation from the input space to the hidden-unit space is nonlinear.

  • On the other hand, the transformation from the hidden space to the output space is linear.

Mahaneeya Raman - Extended Class Project


Benefits of sensewear

Benefits of SenseWear

  • Detect energy expenditure during certain non-ambulatory activities

  • Detect increased effort and energy expenditure associated with load carrying.

  • Measure heat produced by the body as a result of basic metabolism and, as well as, all forms of physical activity.

  • Small, unobtrusive, and comfortable to wear.

  • It is not invasive and does not alter normal patterns of motion or activity

Mahaneeya Raman - Extended Class Project


  • Login