environmental awareness
Download
Skip this Video
Download Presentation
ENVIRONMENTAL AWARENESS

Loading in 2 Seconds...

play fullscreen
1 / 35

ENVIRONMENTAL AWARENESS - PowerPoint PPT Presentation


  • 94 Views
  • Uploaded on

ENVIRONMENTAL AWARENESS. Mahaneeya Raman . Automated Activity and Object Detection from Soldier-worn sensors. To monitor and assist soldiers Monitor environment and soldier entity Data acquisition through: Omni-vision Camera Body sensors GPS/GIS. Agenda.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ENVIRONMENTAL AWARENESS' - atalanta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
environmental awareness

ENVIRONMENTAL AWARENESS

Mahaneeya Raman

Mahaneeya Raman - Extended Class Project

automated activity and object detection from soldier worn sensors
Automated Activity and Object Detection from Soldier-worn sensors
  • To monitor and assist soldiers
  • Monitor environment and soldier entity
  • Data acquisition through:
    • Omni-vision Camera
    • Body sensors
    • GPS/GIS

Mahaneeya Raman - Extended Class Project

agenda
Agenda
  • Brief overview of Omni-vision Camera System
  • Brief overview of body sensors
  • Overview of GPS/GIS
  • Image Stitching –procedure and problems encountered
  • BodyMedia SenseWear sensor system

Mahaneeya Raman - Extended Class Project

omni vision camera system
Omni-vision Camera System
  • Wide angle lenses (f= 1.7 mm ~ 110 degrees)
  • Higher image resolution than mirror cameras and cheaper ones.
  • Cameras daisy chained.
  • Just 1 Fire wire cable (signal +power)

Mahaneeya Raman - Extended Class Project

body sensors to predict physical activity and emotion
Body sensors – to predict physical activity and emotion
  • Physiological data related to human activity and emotion
  • Physical activity – accelerometer signals, temperature, heat flux
  • Human emotion – Skin conductance (Galvanic Skin Response), heart rate

Mahaneeya Raman - Extended Class Project

gps gis
GPS/GIS
  • Map data to physical coordinates
  • Simplify the classification problem – index into previously collected databases
  • For visualization purposes – need Geographic Information System (GIS) data
  • Setback – extensive GIS data is not free!!

Mahaneeya Raman - Extended Class Project

image stitching steps involved
Image Stitching – steps involved
  • Camera Calibration
  • Correction for radial and tangential distortion
  • Projection onto cylindrical coordinates
  • Image stitching
  • Take-away from this work:
    • Problems faced - Parallax error
    • Suggestion on how to solve

Mahaneeya Raman - Extended Class Project

camera calibration
Camera Calibration
  • To correct radial distortion and tangential distortion in images
  • Radial – Straight lines in real world appear curved in the image plane
  • Tangential – Image not located on a strict plane surface

Mahaneeya Raman - Extended Class Project

undistorting image and projection onto cylindrical coordinates
Undistorting image and projection onto cylindrical coordinates
  • Polynomial distortion model
  • For aligning the images
  • To fit together as a panorama
  • Conversion from Cartesian to Cylindrical coordinates:

r2 = x2 + y2

tan θ = y / x

Mahaneeya Raman - Extended Class Project

images
Images

Distorted Image Corrected image and

projected onto

cylindrical coordinates

Mahaneeya Raman - Extended Class Project

image stitching
Image Stitching
  • Load the two undistorted images
  • Select points using

cpselect function in Matlab. Frame1_points and Frame2_points saved.

  • Determine transformation

T, between the 2 images,

x’ = Tx

  • Map one image onto the other based on the transformation.

Mahaneeya Raman - Extended Class Project

image stitching output
Image Stitching output

Final output

Mahaneeya Raman - Extended Class Project

image stitching of raw images parallax error
Image stitching of raw images – Parallax error

Mahaneeya Raman - Extended Class Project

understanding parallax error
Understanding Parallax error
  • Distance related measurement error
  • High for images that are close to the camera lens
  • Occurs because Same points are located at different distances from two camera lenses, in both images
  • Distances between 2 given points is not the same in both images
  • Solution :
    • non-linear mapping has to be done
    • Ex. Radial Basis Function method / full planar perspective models

Mahaneeya Raman - Extended Class Project

bodymedia sensewear pro 2 armband
BodyMedia® SenseWear PRO2 Armband
  • Body sensor worn on upper right arm
  • Timestamp, Memory, Battery indicators
  • Connects to PC through USB cable
  • Collects through 6 continuous streams of data channels
  • Stores 30 channels of data

Mahaneeya Raman - Extended Class Project

bodymedia sensewear pro 2 armband1
BodyMedia® SenseWear PRO2 Armband
  • 2 axis accelerometer
  • Heat flux sensor
  • Galvanic Skin Response sensor
  • Skin Temperature sensor
  • Near-Body Temperature Sensor

Mahaneeya Raman - Extended Class Project

sensor signal graph for typing writing
Sensor signal graph for typing-writing

Skin Temperature

Longitudinal Accelerometer

Heat Flux

Transverse Accelerometer

Galvanic Skin Response

Mahaneeya Raman - Extended Class Project

slide18

Applications of SenseWear

Mahaneeya Raman - Extended Class Project

data collection
Data Collection
  • Pattern of data collected:
    • Typing – writing
    • Walking – simply sitting
    • Playing ‘quake’ – watching a comedy clip
  • Sample rate - 4 samples/second
  • Video surveillance using fire-i cameras while data is being collected
  • Timestamp between activities

Mahaneeya Raman - Extended Class Project

recorded video using 2 fire i cameras
Recorded video using 2 fire-i cameras

Mahaneeya Raman - Extended Class Project

sensor signal graph for typing writing walking sitting playing quake relaxing
Sensor signal graph for typing-writing, walking - sitting, playing ‘quake’-relaxing

Longitudinal Accelerometer

Transverse Accelerometer

Heat Flux

Mahaneeya Raman - Extended Class Project

data analysis
Data Analysis
  • Generate signal graph and excel sheets using bodymedia’s innerview software
  • Notice Accelerometer signals change considerably
  • Use a classifier algorithm
  • 50% data – training
  • 50% data – testing

Mahaneeya Raman - Extended Class Project

knn and lda classifier considered algorithms for comparison
KNN and LDA classifier – considered Algorithms for comparison
  • KNN – K Nearest Neighbor Algorithm
    • If x – to be classified, and (x1, y1), . . . , (xk, yk) are x’s k nearest neighbors, and d(x, xi) = distance between x and xi, x is classified into the nearest neighbor cluster.
  • LDA
    • Method to find linear discriminant boundaries between K classes
    • Define K linear discriminant functions for K classes
    • Classify x to the class with the largest value for its discriminant function

Mahaneeya Raman - Extended Class Project

conclusion
Conclusion
  • Useful GIS data can be integrated with GPS data for effective localization and environment analysis
  • Parallax error – can be solved by applying a non-linear transformation like RBF or full planar perspective models
  • Pending - Compare classifier algorithms to classify activity, which can be extended to predict human emotion as well

Mahaneeya Raman - Extended Class Project

references
References

[1] CAMEO: The Camera Assisted Meeting Event Observer – Paul E. Rybski, Fernando de la Torre, Raju Patil, Carlos Vallespi, Manuela Veloso, Brett Browning.

[2] Image Warping - Mikkel B. Stegmann , Informatics and Mathematical Modelling, Technical University of Denmark.

[3] Creating Full View Panoramic Image Mosaics and Environment Maps - Richard Szeliski and Heung-Yeung Shum, Microsoft Research.

[4] 16 papers on bodymedia applications - http://www.bodymedia.com/research/whitepapers.jsp

Mahaneeya Raman - Extended Class Project

thank you
Thank you

Mahaneeya Raman - Extended Class Project

matlab toolbox for calibration
Matlab toolbox for calibration
  • Intrinsic calibration – focal length for each image axis, an image center, 3 terms of radial distortion, and 2 terms of tangential distortion.
  • Checker-board method – straight lines with easily localizable end points and interior points can be found in several orientations throughout the image plane.

Mahaneeya Raman - Extended Class Project

matlab toolbox for calibration1
Matlab toolbox for calibration
  • Images of checkerboard at different inclinations (horizontal, vertical, diagonal)
  • Provide size of the square
  • Extract grid corners of all images, one by one.
  • Provide size of window of squares chosen
  • Corner extraction - verification

Mahaneeya Raman - Extended Class Project

distortion model plumb bob model
Distortion Model – “Plumb Bob” model
  • P – point in space coordinate vector,

XXc= [Xc;Yc;Zc] in camera ref. frame

  • Project P on the image plane according to intrinsic parameters (fc, cc, alpha_c, kc)
  • xn – normalized (pinhole) image projection,

xn =[ Xc/ Zc;Yc/ Zc] = [x ; y]

  • Let r2 = x2 + y2

Mahaneeya Raman - Extended Class Project

distortion model plumb bob model1
Distortion Model – “Plumb Bob” model
  • After lens distortion, the new normalized point coordinate, xd = [xd(1) ; xd(2)]

xd = (1+ kc(1)r2 + kc(2)r4 + kc(5)r6 )xn + dx

  • Where, dx – tangential distortion vector

dx = [2kc(3)xy+ kc(4)(r2 + 2x2) ;

kc(3)(r2 + 2y2)+ 2kc(4)xy ]

Mahaneeya Raman - Extended Class Project

distortion model
Distortion Model
  • The final pixel coordinates, x_pixel = [xp;yp]
  • Xp = fc(1)(Xd (1) + alpha_c*Xd (2)) +cc(1)
  • Yp = fc(2) Xd (2) +cc(2)
  • Therefore, [xp;yp;1] = KK [xd(1);xd(2);1]

where, KK = [fc(1) alpha_c*fc(1) cc(1) ;

0 fc(1) cc(1) ;

0 0 1 ]

Mahaneeya Raman - Extended Class Project

image stitching1
Image Stitching
  • Load 2 images, distortion corrected and projected onto cylindrical coordinates.
  • Select points using cpselect function in Matlab. Input_points and base_points saved.
  • Determine transformation T, between the 2 images,

x’ = Tx

x s cosα s sinα tx x

y =-s sinα s cosα ty y

1 0 0 1 1

Mahaneeya Raman - Extended Class Project

image stitching continued
Image Stitching - continued
  • By rearranging the equation so the warping parameters is the vector t in,

x’ = Zt

x’ x y 1 0 0s cosα

y’ =y -x 0 1 0s sinα

1 0 0 0 0 1 tx

ty

1

Mahaneeya Raman - Extended Class Project

radial basis function method
Radial Basis Function method
  • The input layer is the set of source nodes (sensory units).
  • The second layer is a hidden layer of high dimension.
  • The output layer gives the response of the network to the activation patterns applied to the input layer.
  • The transformation from the input space to the hidden-unit space is nonlinear.
  • On the other hand, the transformation from the hidden space to the output space is linear.

Mahaneeya Raman - Extended Class Project

benefits of sensewear
Benefits of SenseWear
  • Detect energy expenditure during certain non-ambulatory activities
  • Detect increased effort and energy expenditure associated with load carrying.
  • Measure heat produced by the body as a result of basic metabolism and, as well as, all forms of physical activity.
  • Small, unobtrusive, and comfortable to wear.
  • It is not invasive and does not alter normal patterns of motion or activity

Mahaneeya Raman - Extended Class Project

ad