1 / 35

ENVIRONMENTAL AWARENESS

ENVIRONMENTAL AWARENESS. Mahaneeya Raman . Automated Activity and Object Detection from Soldier-worn sensors. To monitor and assist soldiers Monitor environment and soldier entity Data acquisition through: Omni-vision Camera Body sensors GPS/GIS. Agenda.

atalanta
Download Presentation

ENVIRONMENTAL AWARENESS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ENVIRONMENTAL AWARENESS Mahaneeya Raman Mahaneeya Raman - Extended Class Project

  2. Automated Activity and Object Detection from Soldier-worn sensors • To monitor and assist soldiers • Monitor environment and soldier entity • Data acquisition through: • Omni-vision Camera • Body sensors • GPS/GIS Mahaneeya Raman - Extended Class Project

  3. Agenda • Brief overview of Omni-vision Camera System • Brief overview of body sensors • Overview of GPS/GIS • Image Stitching –procedure and problems encountered • BodyMedia SenseWear sensor system Mahaneeya Raman - Extended Class Project

  4. Omni-vision Camera System • Wide angle lenses (f= 1.7 mm ~ 110 degrees) • Higher image resolution than mirror cameras and cheaper ones. • Cameras daisy chained. • Just 1 Fire wire cable (signal +power) Mahaneeya Raman - Extended Class Project

  5. Body sensors – to predict physical activity and emotion • Physiological data related to human activity and emotion • Physical activity – accelerometer signals, temperature, heat flux • Human emotion – Skin conductance (Galvanic Skin Response), heart rate Mahaneeya Raman - Extended Class Project

  6. GPS/GIS • Map data to physical coordinates • Simplify the classification problem – index into previously collected databases • For visualization purposes – need Geographic Information System (GIS) data • Setback – extensive GIS data is not free!! Mahaneeya Raman - Extended Class Project

  7. Image Stitching – steps involved • Camera Calibration • Correction for radial and tangential distortion • Projection onto cylindrical coordinates • Image stitching • Take-away from this work: • Problems faced - Parallax error • Suggestion on how to solve Mahaneeya Raman - Extended Class Project

  8. Camera Calibration • To correct radial distortion and tangential distortion in images • Radial – Straight lines in real world appear curved in the image plane • Tangential – Image not located on a strict plane surface Mahaneeya Raman - Extended Class Project

  9. Undistorting image and projection onto cylindrical coordinates • Polynomial distortion model • For aligning the images • To fit together as a panorama • Conversion from Cartesian to Cylindrical coordinates: r2 = x2 + y2 tan θ = y / x Mahaneeya Raman - Extended Class Project

  10. Images Distorted Image Corrected image and projected onto cylindrical coordinates Mahaneeya Raman - Extended Class Project

  11. Image Stitching • Load the two undistorted images • Select points using cpselect function in Matlab. Frame1_points and Frame2_points saved. • Determine transformation T, between the 2 images, x’ = Tx • Map one image onto the other based on the transformation. Mahaneeya Raman - Extended Class Project

  12. Image Stitching output Final output Mahaneeya Raman - Extended Class Project

  13. Image stitching of raw images – Parallax error Mahaneeya Raman - Extended Class Project

  14. Understanding Parallax error • Distance related measurement error • High for images that are close to the camera lens • Occurs because Same points are located at different distances from two camera lenses, in both images • Distances between 2 given points is not the same in both images • Solution : • non-linear mapping has to be done • Ex. Radial Basis Function method / full planar perspective models Mahaneeya Raman - Extended Class Project

  15. BodyMedia® SenseWear PRO2 Armband • Body sensor worn on upper right arm • Timestamp, Memory, Battery indicators • Connects to PC through USB cable • Collects through 6 continuous streams of data channels • Stores 30 channels of data Mahaneeya Raman - Extended Class Project

  16. BodyMedia® SenseWear PRO2 Armband • 2 axis accelerometer • Heat flux sensor • Galvanic Skin Response sensor • Skin Temperature sensor • Near-Body Temperature Sensor Mahaneeya Raman - Extended Class Project

  17. Sensor signal graph for typing-writing Skin Temperature Longitudinal Accelerometer Heat Flux Transverse Accelerometer Galvanic Skin Response Mahaneeya Raman - Extended Class Project

  18. Applications of SenseWear Mahaneeya Raman - Extended Class Project

  19. Data Collection • Pattern of data collected: • Typing – writing • Walking – simply sitting • Playing ‘quake’ – watching a comedy clip • Sample rate - 4 samples/second • Video surveillance using fire-i cameras while data is being collected • Timestamp between activities Mahaneeya Raman - Extended Class Project

  20. Recorded video using 2 fire-i cameras Mahaneeya Raman - Extended Class Project

  21. Sensor signal graph for typing-writing, walking - sitting, playing ‘quake’-relaxing Longitudinal Accelerometer Transverse Accelerometer Heat Flux Mahaneeya Raman - Extended Class Project

  22. Data Analysis • Generate signal graph and excel sheets using bodymedia’s innerview software • Notice Accelerometer signals change considerably • Use a classifier algorithm • 50% data – training • 50% data – testing Mahaneeya Raman - Extended Class Project

  23. KNN and LDA classifier – considered Algorithms for comparison • KNN – K Nearest Neighbor Algorithm • If x – to be classified, and (x1, y1), . . . , (xk, yk) are x’s k nearest neighbors, and d(x, xi) = distance between x and xi, x is classified into the nearest neighbor cluster. • LDA • Method to find linear discriminant boundaries between K classes • Define K linear discriminant functions for K classes • Classify x to the class with the largest value for its discriminant function Mahaneeya Raman - Extended Class Project

  24. Conclusion • Useful GIS data can be integrated with GPS data for effective localization and environment analysis • Parallax error – can be solved by applying a non-linear transformation like RBF or full planar perspective models • Pending - Compare classifier algorithms to classify activity, which can be extended to predict human emotion as well Mahaneeya Raman - Extended Class Project

  25. References [1] CAMEO: The Camera Assisted Meeting Event Observer – Paul E. Rybski, Fernando de la Torre, Raju Patil, Carlos Vallespi, Manuela Veloso, Brett Browning. [2] Image Warping - Mikkel B. Stegmann , Informatics and Mathematical Modelling, Technical University of Denmark. [3] Creating Full View Panoramic Image Mosaics and Environment Maps - Richard Szeliski and Heung-Yeung Shum, Microsoft Research. [4] 16 papers on bodymedia applications - http://www.bodymedia.com/research/whitepapers.jsp Mahaneeya Raman - Extended Class Project

  26. Thank you Mahaneeya Raman - Extended Class Project

  27. Matlab toolbox for calibration • Intrinsic calibration – focal length for each image axis, an image center, 3 terms of radial distortion, and 2 terms of tangential distortion. • Checker-board method – straight lines with easily localizable end points and interior points can be found in several orientations throughout the image plane. Mahaneeya Raman - Extended Class Project

  28. Matlab toolbox for calibration • Images of checkerboard at different inclinations (horizontal, vertical, diagonal) • Provide size of the square • Extract grid corners of all images, one by one. • Provide size of window of squares chosen • Corner extraction - verification Mahaneeya Raman - Extended Class Project

  29. Distortion Model – “Plumb Bob” model • P – point in space coordinate vector, XXc= [Xc;Yc;Zc] in camera ref. frame • Project P on the image plane according to intrinsic parameters (fc, cc, alpha_c, kc) • xn – normalized (pinhole) image projection, xn =[ Xc/ Zc;Yc/ Zc] = [x ; y] • Let r2 = x2 + y2 Mahaneeya Raman - Extended Class Project

  30. Distortion Model – “Plumb Bob” model • After lens distortion, the new normalized point coordinate, xd = [xd(1) ; xd(2)] xd = (1+ kc(1)r2 + kc(2)r4 + kc(5)r6 )xn + dx • Where, dx – tangential distortion vector dx = [2kc(3)xy+ kc(4)(r2 + 2x2) ; kc(3)(r2 + 2y2)+ 2kc(4)xy ] Mahaneeya Raman - Extended Class Project

  31. Distortion Model • The final pixel coordinates, x_pixel = [xp;yp] • Xp = fc(1)(Xd (1) + alpha_c*Xd (2)) +cc(1) • Yp = fc(2) Xd (2) +cc(2) • Therefore, [xp;yp;1] = KK [xd(1);xd(2);1] where, KK = [fc(1) alpha_c*fc(1) cc(1) ; 0 fc(1) cc(1) ; 0 0 1 ] Mahaneeya Raman - Extended Class Project

  32. Image Stitching • Load 2 images, distortion corrected and projected onto cylindrical coordinates. • Select points using cpselect function in Matlab. Input_points and base_points saved. • Determine transformation T, between the 2 images, x’ = Tx x s cosα s sinα tx x y =-s sinα s cosα ty y 1 0 0 1 1 Mahaneeya Raman - Extended Class Project

  33. Image Stitching - continued • By rearranging the equation so the warping parameters is the vector t in, x’ = Zt x’ x y 1 0 0s cosα y’ =y -x 0 1 0s sinα 1 0 0 0 0 1 tx ty 1 Mahaneeya Raman - Extended Class Project

  34. Radial Basis Function method • The input layer is the set of source nodes (sensory units). • The second layer is a hidden layer of high dimension. • The output layer gives the response of the network to the activation patterns applied to the input layer. • The transformation from the input space to the hidden-unit space is nonlinear. • On the other hand, the transformation from the hidden space to the output space is linear. Mahaneeya Raman - Extended Class Project

  35. Benefits of SenseWear • Detect energy expenditure during certain non-ambulatory activities • Detect increased effort and energy expenditure associated with load carrying. • Measure heat produced by the body as a result of basic metabolism and, as well as, all forms of physical activity. • Small, unobtrusive, and comfortable to wear. • It is not invasive and does not alter normal patterns of motion or activity Mahaneeya Raman - Extended Class Project

More Related