slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Smart Meeting Systems PowerPoint Presentation
Download Presentation
Smart Meeting Systems

Loading in 2 Seconds...

play fullscreen
1 / 40
yauvani

Smart Meeting Systems - PowerPoint PPT Presentation

172 Views
Download Presentation
Smart Meeting Systems
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Smart Meeting Systems Josh Reilly

  2. Why are Smart Meeting Systems worth studying?

  3. Objectives of a Smart Meeting System • Improves the productivity of a team by automating the: • Capture of the meeting • Processing of the meeting for valuable information • Displaying of that information accurately and effectively to the end user through a client application

  4. Organization of Smart Meeting System Processes A smart meeting system can be decomposed into three sets of processes • Meeting Capture • Meeting Recognition • Semantic Processing

  5. Organization of Smart Meeting System Processes

  6. Meeting Capture • Gathering raw inputs from the meeting • Video Capture • Audio Capture • Other Context

  7. Video Capture • Video feeds from: • Cameras for the attendees • Could use a single static camera • Could use a single camera with pan, tilt, zoom (PTZ) capabilities • Recommend camera view of every contributor's face • Visual Aids • Separate camera • Digital feed from device

  8. Microsoft Distributed Meetings Project Camera Placement

  9. Microsoft Distributed Meetings ProjectVideo Capture RingCam • Array of 90º Cameras • 360º Panoramic view

  10. Audio Capture • Use an array of microphones • Placed on the table • Placed on the ceiling • Worn on the person • Levels need to be controlled so that they are similar levels for each contributor

  11. Microsoft Distributed Meetings Project Audio Capture • RingCam • Has an array of microphones on its base.

  12. Other Context Capture • RFID to track attendees • Attendees swipe their RFID cards when they enter the meeting to add their ID to the list of people attending this meeting • Motion Detectors • to track the locations of attendees within the room

  13. Organization of Smart Meeting System Processes

  14. Meeting Recognition • The processing of the raw capture before it is organized into something useful • Steps: • Person Identification • Attention Detection • Activity Recognition • Hot Spot Recognition • Summarization

  15. Person Identification • Person Identification is associating sections of video, audio, and the visual aids that were captured from the meeting with the attendee(s) that they belong to • Face Recognition • Face Tracking • Speech Recognition • SSL • Beamforming

  16. Person IdentificationFace Recognition • Facial Recognition • Identify the person speaking from a list of attendees • Eigenface Approach • Challenges • Poor Quality Images • Poor Room Lighting • Continuously changing facial expressions • Occlusion

  17. Face RecognitionThe Eigenface Approach • All faces are assumed to be made up of different percentages of different eigenfaces • A set of eigenfaces is a set of very generalized pictures of faces that were generated so that each has a basic ingredient that can be used to make a face Eigenfaces from AT&T Laboratories Cambridge

  18. Person IdentificationSpeech Recognition • Speech Recognition • Match the voice of the person speaking to someone on the list of attendees • Using Voice recognition in conjunction with face recognition allows for an accurate identification of the speaker • Sound Source Localization (SSL) • Used to determine which camera is pointed at the speaker • Could be used to point PTZ camera • Beamforming

  19. Person IdentificationWriter Recognition • Writer Recognition • When someone writes on the whiteboard, they may not be in clear view of the cameras • Writing recognition algorithms can be used to identify who wrote what during a meeting

  20. Attention Detection • Attention Detection • Attempt to determine who is looking at whom during a meeting. • Provides information used for activity recognition and hot spot recognition • Done using: • Hidden Markov Models (HMM) • Sound Source Localization (SSL) • Known layout of room

  21. Activity Recognition • Determine what is happening during the meeting • Step 1: • Determine what each individual is doing at each point during the meeting • Person Identification, Attention Detection, SSL, Gesture Recognition • Step 2: • Take that information to determine what activity the entire group is engaging in at each point during the meeting

  22. Hot Spot Recognition • Find the important parts of the meeting • Using sound queues • Ex: Changes in pitch • Using activity recognition • When people are nodding • When their focus changes

  23. Summarization • Takes all of the information that the smart meeting system has learned about the meeting and creates a quick overview of the events that took place during that meeting. • This information will be used in the semantic processing stage

  24. Organization of Smart Meeting System Processes

  25. Semantic Processing • Takes the information from the meeting recognition step and makes it usable by the end user. • Meeting Annotation • Meeting Indexing • Meeting Browsing

  26. Meeting Annotation • Describe the raw data from the meeting from each viewpoint • Attempt to label all meeting segments • Implicitly • Automatically • Explicitly • By Hand

  27. Meeting AnnotationImplicit • Automated Annotation • Assumes that the meeting recognition processes performed with relatively high efficiency • Tags every person in the video • Narrates what was happening during the meeting • Has not been achieved

  28. Meeting AnnotationExplicit • Annotation By Hand • When the recognition processes fail to gather sufficient correct information about the raw data • Users will have to go through the meeting and tag the people attending as well as indicate what events are happening all through the meeting

  29. Meeting Indexing • Indexing is done at all levels of data from a raw audio feed to the annotations • The best form of indexing to use is the event-based indexing • An index is created every time an event occurs • This is the best way for users to find a specific spot in the meeting when performing a query

  30. Meeting Browsing • The interface that the end user uses to retrieve information from the meetings • Functions: • Can browse/search a list of all meetings for a specific meeting • Can browse/search the contents of the chosen meeting • Aided by tools like bookmarks, a meeting outline, and queries (content, people, camera angles, visual aids, etc...)

  31. Meeting BrowsingMicrosoft Distributed Meetings

  32. Remote Attendee • Use the smart meeting system as the attendee's eyes and ears • Microsoft's PING project • Uses a monitor and speaker to display the remote attendee's voice and audio during the meeting • However, the remote attendee is often ignored

  33. Carnegie Mellon University’sMeeting System Architecture Lacks • Activity Recognition • Hot Spot Recognition • Annotations

  34. University of California, San DiegoAVIARY System Architecture • 2 PCs • 4 Static Cameras • 4 PTZ Cameras • No SSL

  35. RicohPortable Meeting Recorder

  36. RicohPortable Meeting RecorderDoughnut Camera

  37. RicohPortable Meeting RecorderMeeting Browser

  38. Technology Limitations • Speech recognition and facial recognition algorithms are not yet as efficient as they should be in order for a smart meeting system to perform accurately

  39. Workspace Limitations • Cameras and microphones can block view, distract, or intimidate attendees during the meeting • Security and Privacy needs to be addressed

  40. References [1] Zhiwen Yu and Yuichi Nakamura. 2010. Smart meeting systems: A survey of state-of-the-art and open issues. ACM Comput. Surv. 42, 2, Article 8 (March 2010), 20 pages. DOI=10.1145/1667062.1667065 http://doi.acm.org/10.1145/1667062.1667065 [2] Ross Cutler , Yong Rui , Anoop Gupta , Jj Cadiz , Ivan Tashev , Li-wei He , Alex Colburn , Zhengyou Zhang , Zicheng Liu , Steve Silverberg. (2002). Distributed Meetings. A Meeting Capture and Broadcasting System. 10 pages. http://research.microsoft.com/en-us/um/people/yongrui/ps/mm02.pdf [3] Harold Fox. 2004. The eFacilitator: A Meeting Capture Application and Infrastructure. 89 pages. http://hdl.handle.net/1721.1/17672 [4] Yong Rui, Eric Rudolph, Li-wei He, Rico Malvar, Michael Cohen, Ivan Tashev. 2006. Ping: A Group-To-Individual Distributed meeting System. 4 pages. http://research.microsoft.com/apps/pubs/default.aspx?id=76779 • [5] Dar-Shyang Lee, Berna Erol, Jamey Graham, Jonathan Hull, Norihiko Murata. 2011. Portable Meeting Recorder. 10 pages. http://rii.ricoh.com/sites/default/files/Portable_Meeting_Recorder.pdf