- 47 Views
- Uploaded on
- Presentation posted in: General

ENVIRONMENTAL AWARENESS

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

ENVIRONMENTAL AWARENESS

Mahaneeya Raman

Mahaneeya Raman - Extended Class Project

- To monitor and assist soldiers
- Monitor environment and soldier entity
- Data acquisition through:
- Omni-vision Camera
- Body sensors
- GPS/GIS

Mahaneeya Raman - Extended Class Project

- Brief overview of Omni-vision Camera System
- Brief overview of body sensors
- Overview of GPS/GIS
- Image Stitching –procedure and problems encountered
- BodyMedia SenseWear sensor system

Mahaneeya Raman - Extended Class Project

- Wide angle lenses (f= 1.7 mm ~ 110 degrees)
- Higher image resolution than mirror cameras and cheaper ones.
- Cameras daisy chained.
- Just 1 Fire wire cable (signal +power)

Mahaneeya Raman - Extended Class Project

- Physiological data related to human activity and emotion
- Physical activity – accelerometer signals, temperature, heat flux
- Human emotion – Skin conductance (Galvanic Skin Response), heart rate

Mahaneeya Raman - Extended Class Project

- Map data to physical coordinates
- Simplify the classification problem – index into previously collected databases
- For visualization purposes – need Geographic Information System (GIS) data
- Setback – extensive GIS data is not free!!

Mahaneeya Raman - Extended Class Project

- Camera Calibration
- Correction for radial and tangential distortion
- Projection onto cylindrical coordinates
- Image stitching
- Take-away from this work:
- Problems faced - Parallax error
- Suggestion on how to solve

Mahaneeya Raman - Extended Class Project

- To correct radial distortion and tangential distortion in images
- Radial – Straight lines in real world appear curved in the image plane
- Tangential – Image not located on a strict plane surface

Mahaneeya Raman - Extended Class Project

- Polynomial distortion model
- For aligning the images
- To fit together as a panorama
- Conversion from Cartesian to Cylindrical coordinates:
r2 = x2 + y2

tan θ = y / x

Mahaneeya Raman - Extended Class Project

Distorted ImageCorrected image and

projected onto

cylindrical coordinates

Mahaneeya Raman - Extended Class Project

- Load the two undistorted images
- Select points using
cpselect function in Matlab. Frame1_points and Frame2_points saved.

- Determine transformation
T, between the 2 images,

x’ = Tx

- Map one image onto the other based on the transformation.

Mahaneeya Raman - Extended Class Project

Final output

Mahaneeya Raman - Extended Class Project

Mahaneeya Raman - Extended Class Project

- Distance related measurement error
- High for images that are close to the camera lens
- Occurs because Same points are located at different distances from two camera lenses, in both images
- Distances between 2 given points is not the same in both images
- Solution :
- non-linear mapping has to be done
- Ex. Radial Basis Function method / full planar perspective models

Mahaneeya Raman - Extended Class Project

- Body sensor worn on upper right arm
- Timestamp, Memory, Battery indicators
- Connects to PC through USB cable
- Collects through 6 continuous streams of data channels
- Stores 30 channels of data

Mahaneeya Raman - Extended Class Project

- 2 axis accelerometer
- Heat flux sensor
- Galvanic Skin Response sensor
- Skin Temperature sensor
- Near-Body Temperature Sensor

Mahaneeya Raman - Extended Class Project

Skin Temperature

Longitudinal Accelerometer

Heat Flux

Transverse Accelerometer

Galvanic Skin Response

Mahaneeya Raman - Extended Class Project

Applications of SenseWear

Mahaneeya Raman - Extended Class Project

- Pattern of data collected:
- Typing – writing
- Walking – simply sitting
- Playing ‘quake’ – watching a comedy clip

- Sample rate - 4 samples/second
- Video surveillance using fire-i cameras while data is being collected
- Timestamp between activities

Mahaneeya Raman - Extended Class Project

Mahaneeya Raman - Extended Class Project

Longitudinal Accelerometer

Transverse Accelerometer

Heat Flux

Mahaneeya Raman - Extended Class Project

- Generate signal graph and excel sheets using bodymedia’s innerview software
- Notice Accelerometer signals change considerably
- Use a classifier algorithm
- 50% data – training
- 50% data – testing

Mahaneeya Raman - Extended Class Project

- KNN – K Nearest Neighbor Algorithm
- If x – to be classified, and (x1, y1), . . . , (xk, yk) are x’s k nearest neighbors, and d(x, xi) = distance between x and xi, x is classified into the nearest neighbor cluster.

- LDA
- Method to find linear discriminant boundaries between K classes
- Define K linear discriminant functions for K classes
- Classify x to the class with the largest value for its discriminant function

Mahaneeya Raman - Extended Class Project

- Useful GIS data can be integrated with GPS data for effective localization and environment analysis
- Parallax error – can be solved by applying a non-linear transformation like RBF or full planar perspective models
- Pending - Compare classifier algorithms to classify activity, which can be extended to predict human emotion as well

Mahaneeya Raman - Extended Class Project

[1]CAMEO: The Camera Assisted Meeting Event Observer – Paul E. Rybski, Fernando de la Torre, Raju Patil, Carlos Vallespi, Manuela Veloso, Brett Browning.

[2]Image Warping - Mikkel B. Stegmann , Informatics and Mathematical Modelling, Technical University of Denmark.

[3]Creating Full View Panoramic Image Mosaics and Environment Maps - Richard Szeliski and Heung-Yeung Shum, Microsoft Research.

[4]16 papers on bodymedia applications - http://www.bodymedia.com/research/whitepapers.jsp

Mahaneeya Raman - Extended Class Project

Mahaneeya Raman - Extended Class Project

- Intrinsic calibration – focal length for each image axis, an image center, 3 terms of radial distortion, and 2 terms of tangential distortion.
- Checker-board method – straight lines with easily localizable end points and interior points can be found in several orientations throughout the image plane.

Mahaneeya Raman - Extended Class Project

- Images of checkerboard at different inclinations (horizontal, vertical, diagonal)
- Provide size of the square
- Extract grid corners of all images, one by one.
- Provide size of window of squares chosen
- Corner extraction - verification

Mahaneeya Raman - Extended Class Project

- P – point in space coordinate vector,
XXc= [Xc;Yc;Zc] in camera ref. frame

- Project P on the image plane according to intrinsic parameters (fc, cc, alpha_c, kc)
- xn – normalized (pinhole) image projection,
xn =[ Xc/ Zc;Yc/ Zc] = [x ; y]

- Let r2 = x2 + y2

Mahaneeya Raman - Extended Class Project

- After lens distortion, the new normalized point coordinate, xd = [xd(1) ; xd(2)]
xd = (1+ kc(1)r2 + kc(2)r4 + kc(5)r6 )xn + dx

- Where, dx – tangential distortion vector
dx = [2kc(3)xy+ kc(4)(r2 + 2x2) ;

kc(3)(r2 + 2y2)+ 2kc(4)xy ]

Mahaneeya Raman - Extended Class Project

- The final pixel coordinates, x_pixel = [xp;yp]
- Xp = fc(1)(Xd (1) + alpha_c*Xd (2)) +cc(1)
- Yp = fc(2) Xd (2) +cc(2)
- Therefore, [xp;yp;1] = KK [xd(1);xd(2);1]
where, KK = [fc(1)alpha_c*fc(1)cc(1) ;

0fc(1)cc(1) ;

001]

Mahaneeya Raman - Extended Class Project

- Load 2 images, distortion corrected and projected onto cylindrical coordinates.
- Select points using cpselect function in Matlab. Input_points and base_points saved.
- Determine transformation T, between the 2 images,
x’ = Tx

x s cosα s sinα tx x

y =-s sinα s cosα tyy

1001 1

Mahaneeya Raman - Extended Class Project

- By rearranging the equation so the warping parameters is the vector t in,
x’ = Zt

x’xy100s cosα

y’ =y-x010s sinα

1000 01 tx

ty

1

Mahaneeya Raman - Extended Class Project

- The input layer is the set of source nodes (sensory units).
- The second layer is a hidden layer of high dimension.
- The output layer gives the response of the network to the activation patterns applied to the input layer.
- The transformation from the input space to the hidden-unit space is nonlinear.
- On the other hand, the transformation from the hidden space to the output space is linear.

Mahaneeya Raman - Extended Class Project

- Detect energy expenditure during certain non-ambulatory activities
- Detect increased effort and energy expenditure associated with load carrying.
- Measure heat produced by the body as a result of basic metabolism and, as well as, all forms of physical activity.
- Small, unobtrusive, and comfortable to wear.
- It is not invasive and does not alter normal patterns of motion or activity

Mahaneeya Raman - Extended Class Project