1 / 40

Smart Meeting Systems

Smart Meeting Systems. Josh Reilly. Why are Smart Meeting Systems worth studying?. Objectives of a Smart Meeting System. Improves the productivity of a team by automating the: Capture of the meeting Processing of the meeting for valuable information

yauvani
Download Presentation

Smart Meeting Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Smart Meeting Systems Josh Reilly

  2. Why are Smart Meeting Systems worth studying?

  3. Objectives of a Smart Meeting System • Improves the productivity of a team by automating the: • Capture of the meeting • Processing of the meeting for valuable information • Displaying of that information accurately and effectively to the end user through a client application

  4. Organization of Smart Meeting System Processes A smart meeting system can be decomposed into three sets of processes • Meeting Capture • Meeting Recognition • Semantic Processing

  5. Organization of Smart Meeting System Processes

  6. Meeting Capture • Gathering raw inputs from the meeting • Video Capture • Audio Capture • Other Context

  7. Video Capture • Video feeds from: • Cameras for the attendees • Could use a single static camera • Could use a single camera with pan, tilt, zoom (PTZ) capabilities • Recommend camera view of every contributor's face • Visual Aids • Separate camera • Digital feed from device

  8. Microsoft Distributed Meetings Project Camera Placement

  9. Microsoft Distributed Meetings ProjectVideo Capture RingCam • Array of 90º Cameras • 360º Panoramic view

  10. Audio Capture • Use an array of microphones • Placed on the table • Placed on the ceiling • Worn on the person • Levels need to be controlled so that they are similar levels for each contributor

  11. Microsoft Distributed Meetings Project Audio Capture • RingCam • Has an array of microphones on its base.

  12. Other Context Capture • RFID to track attendees • Attendees swipe their RFID cards when they enter the meeting to add their ID to the list of people attending this meeting • Motion Detectors • to track the locations of attendees within the room

  13. Organization of Smart Meeting System Processes

  14. Meeting Recognition • The processing of the raw capture before it is organized into something useful • Steps: • Person Identification • Attention Detection • Activity Recognition • Hot Spot Recognition • Summarization

  15. Person Identification • Person Identification is associating sections of video, audio, and the visual aids that were captured from the meeting with the attendee(s) that they belong to • Face Recognition • Face Tracking • Speech Recognition • SSL • Beamforming

  16. Person IdentificationFace Recognition • Facial Recognition • Identify the person speaking from a list of attendees • Eigenface Approach • Challenges • Poor Quality Images • Poor Room Lighting • Continuously changing facial expressions • Occlusion

  17. Face RecognitionThe Eigenface Approach • All faces are assumed to be made up of different percentages of different eigenfaces • A set of eigenfaces is a set of very generalized pictures of faces that were generated so that each has a basic ingredient that can be used to make a face Eigenfaces from AT&T Laboratories Cambridge

  18. Person IdentificationSpeech Recognition • Speech Recognition • Match the voice of the person speaking to someone on the list of attendees • Using Voice recognition in conjunction with face recognition allows for an accurate identification of the speaker • Sound Source Localization (SSL) • Used to determine which camera is pointed at the speaker • Could be used to point PTZ camera • Beamforming

  19. Person IdentificationWriter Recognition • Writer Recognition • When someone writes on the whiteboard, they may not be in clear view of the cameras • Writing recognition algorithms can be used to identify who wrote what during a meeting

  20. Attention Detection • Attention Detection • Attempt to determine who is looking at whom during a meeting. • Provides information used for activity recognition and hot spot recognition • Done using: • Hidden Markov Models (HMM) • Sound Source Localization (SSL) • Known layout of room

  21. Activity Recognition • Determine what is happening during the meeting • Step 1: • Determine what each individual is doing at each point during the meeting • Person Identification, Attention Detection, SSL, Gesture Recognition • Step 2: • Take that information to determine what activity the entire group is engaging in at each point during the meeting

  22. Hot Spot Recognition • Find the important parts of the meeting • Using sound queues • Ex: Changes in pitch • Using activity recognition • When people are nodding • When their focus changes

  23. Summarization • Takes all of the information that the smart meeting system has learned about the meeting and creates a quick overview of the events that took place during that meeting. • This information will be used in the semantic processing stage

  24. Organization of Smart Meeting System Processes

  25. Semantic Processing • Takes the information from the meeting recognition step and makes it usable by the end user. • Meeting Annotation • Meeting Indexing • Meeting Browsing

  26. Meeting Annotation • Describe the raw data from the meeting from each viewpoint • Attempt to label all meeting segments • Implicitly • Automatically • Explicitly • By Hand

  27. Meeting AnnotationImplicit • Automated Annotation • Assumes that the meeting recognition processes performed with relatively high efficiency • Tags every person in the video • Narrates what was happening during the meeting • Has not been achieved

  28. Meeting AnnotationExplicit • Annotation By Hand • When the recognition processes fail to gather sufficient correct information about the raw data • Users will have to go through the meeting and tag the people attending as well as indicate what events are happening all through the meeting

  29. Meeting Indexing • Indexing is done at all levels of data from a raw audio feed to the annotations • The best form of indexing to use is the event-based indexing • An index is created every time an event occurs • This is the best way for users to find a specific spot in the meeting when performing a query

  30. Meeting Browsing • The interface that the end user uses to retrieve information from the meetings • Functions: • Can browse/search a list of all meetings for a specific meeting • Can browse/search the contents of the chosen meeting • Aided by tools like bookmarks, a meeting outline, and queries (content, people, camera angles, visual aids, etc...)

  31. Meeting BrowsingMicrosoft Distributed Meetings

  32. Remote Attendee • Use the smart meeting system as the attendee's eyes and ears • Microsoft's PING project • Uses a monitor and speaker to display the remote attendee's voice and audio during the meeting • However, the remote attendee is often ignored

  33. Carnegie Mellon University’sMeeting System Architecture Lacks • Activity Recognition • Hot Spot Recognition • Annotations

  34. University of California, San DiegoAVIARY System Architecture • 2 PCs • 4 Static Cameras • 4 PTZ Cameras • No SSL

  35. RicohPortable Meeting Recorder

  36. RicohPortable Meeting RecorderDoughnut Camera

  37. RicohPortable Meeting RecorderMeeting Browser

  38. Technology Limitations • Speech recognition and facial recognition algorithms are not yet as efficient as they should be in order for a smart meeting system to perform accurately

  39. Workspace Limitations • Cameras and microphones can block view, distract, or intimidate attendees during the meeting • Security and Privacy needs to be addressed

  40. References [1] Zhiwen Yu and Yuichi Nakamura. 2010. Smart meeting systems: A survey of state-of-the-art and open issues. ACM Comput. Surv. 42, 2, Article 8 (March 2010), 20 pages. DOI=10.1145/1667062.1667065 http://doi.acm.org/10.1145/1667062.1667065 [2] Ross Cutler , Yong Rui , Anoop Gupta , Jj Cadiz , Ivan Tashev , Li-wei He , Alex Colburn , Zhengyou Zhang , Zicheng Liu , Steve Silverberg. (2002). Distributed Meetings. A Meeting Capture and Broadcasting System. 10 pages. http://research.microsoft.com/en-us/um/people/yongrui/ps/mm02.pdf [3] Harold Fox. 2004. The eFacilitator: A Meeting Capture Application and Infrastructure. 89 pages. http://hdl.handle.net/1721.1/17672 [4] Yong Rui, Eric Rudolph, Li-wei He, Rico Malvar, Michael Cohen, Ivan Tashev. 2006. Ping: A Group-To-Individual Distributed meeting System. 4 pages. http://research.microsoft.com/apps/pubs/default.aspx?id=76779 • [5] Dar-Shyang Lee, Berna Erol, Jamey Graham, Jonathan Hull, Norihiko Murata. 2011. Portable Meeting Recorder. 10 pages. http://rii.ricoh.com/sites/default/files/Portable_Meeting_Recorder.pdf

More Related