1 / 17

Multimodal Assisted Living Environment

Multimodal Assisted Living Environment. Lale Akarun , Alexey Karpov , Hülya Yalçın, Alexander Ronzhin , İsmail Arı, Barış Evrim Demiröz , Aysun Çoban. August2011, eNTERFACE’11 Summer Workshop, Plzen. Outline. System Components. Sample room setup. Table 1. Mic. Mic. Table 2.

sorley
Download Presentation

Multimodal Assisted Living Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimodal Assisted Living Environment Lale Akarun, AlexeyKarpov, Hülya Yalçın, AlexanderRonzhin, İsmail Arı, Barış Evrim Demiröz, Aysun Çoban August2011, eNTERFACE’11 Summer Workshop, Plzen

  2. Outline System Components Multimodal Assisted Living Environment

  3. Sample room setup Table 1 Mic Mic Table 2 Side Cam Chair 1 Chair 2 Up Cam Mic Mic Sink Door Multimodal Assisted Living Environment

  4. Sample scenarios See the videos Multimodal Assisted Living Environment

  5. Automatic speech recognition Detect commands and emergency situations Multimodal Assisted Living Environment

  6. Sound devices setup • Microphone arrays • M-Audio ProFire 2626 Multimodal Assisted Living Environment

  7. Speech recognition samples Answer Phone Help Cough Key Dropping Moan (Cry after Fall) Chair Moving • Voice Activity Detection(VAD) • Speech Commands • Acoustic Event Detection (AED) • Acoustic examples Multimodal Assisted Living Environment

  8. Whole audiodata set (22 events) • 5 English: • Yes • No • Answer Phone • Help • Problem • 5 Russian: • Da • Net • Otvetit • Pomogite • Problema • 12 acoustic: • Applause • Chair Moving • Cough • Cry (Moan) • Door (open/close) • Fall • Key dropping • Key jingle • Paper Work • Phone Ring • Step • Throat cleaning. See the videos Multimodal Assisted Living Environment

  9. Experimental results for VAD Multimodal Assisted Living Environment

  10. Experimental results for AED Multimodal Assisted Living Environment

  11. Background/Foreground Segmentation • Shadow removal Mixture of Gaussians Multimodal Assisted Living Environment

  12. Blob detection • Threshold the areas • filter small contours • Morphological close • maintain connectivity × × See the videos Multimodal Assisted Living Environment

  13. Feature Point Detection and Descriptor Extraction • FAST* corner detection • BRIEF features • extended for color images • *Features from Accelerated Segment Test Multimodal Assisted Living Environment

  14. Blob Matching and Tracking ? person A person C person B See the videos Most feature correspondence Multimodal Assisted Living Environment

  15. Blob Matching and Tracking Frame Blob Blob Blob … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … Keypoint Keypoint Keypoint Multimodal Assisted Living Environment

  16. Blob Matching and Tracking Frame Frame Frame Frame … Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … … … … … … … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Multimodal Assisted Living Environment

More Related