1 / 17

ARCHIVUS: A System for Accessing the Content of Recorded Multimodal Meetings

ARCHIVUS: A System for Accessing the Content of Recorded Multimodal Meetings. Agnes Lisowska Martin Rajman Trung H. Bui IM2.MDM. Introduction.

trish
Download Presentation

ARCHIVUS: A System for Accessing the Content of Recorded Multimodal Meetings

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ARCHIVUS:A System for Accessing the Content of Recorded Multimodal Meetings Agnes Lisowska Martin Rajman Trung H. Bui IM2.MDM

  2. Introduction • Goal: design and develop a system that allows users to access and retrieve the content of stored multimodal meetings, either through directed search or browsing, in a way that is most comfortable to them.

  3. Approach to Design • Need a novel approach to HCI design for this domain – old approaches aren’t applicable • We chose a strongly user-driven approach • Based on user requirements analysis (Lisowska 03) • Considering 5 use cases/scenarios • Manager tracking project progress • Manager tracking employee performance • New employee learning about a project • Employee who missed a meeting they should have been at • Confirmation of facts • Focus on desktop environment

  4. The ARCHIVUS Metaphor • Person using an archive or a library • Each meeting is represented by a book

  5. The Meeting Representation • Example of a page in the book

  6. The ARCHIVUS System • Composed of three tiers

  7. Modalities in ARCHIVUS • Input modalities • 3 active modalities • Voice, pointing (mouse or touch screen), text (keyboard) • 1 passive modality • Emotion recognition (based on facial expression recognition) • Output modalities • Graphics, sound, text

  8. The UI – Search Functionality • Allows the user to initiate a search • Voice, text and pointing modalities systematically associated and always available • Uses system outputs (e.g. prompts) to guide the interaction • Two primary elements • Information request area • allows vocal and textual input as well as additional multimodal interaction through an interactive layout • Information request refinement buttons

  9. The UI – An Example Prompt: Please specify a location. Text: Person Location Date In focus Active Not active

  10. The UI – Interaction History • Shows the constraints that have been imposed on the system so far • Informs the user which variable values are currently in use/bound • Icons with some accompanying text • Scrollable pane, with items in reverse chronological order

  11. The UI – Displaying Results • Results can be displayed on two levels • Global (meeting level) • Bookcase (if more than one meeting is a match) • Arrangement of books reflects the various criteria used and frequency of their occurrences • Book (if only one meeting is a match) • Local (part of a specific meeting) • Pages of a book

  12. The UI – Viewing Area • Detailed view of multimedia elements • VCR style layout for playing video • Radio style layout for audio • Web-style layout for viewing accompanying documents • Allows for interactive linking with book

  13. The Dialogue Manager

  14. The Task/Data Model Solution Table

  15. ARCHIVUS Data • Audio, video and annotated transcripts • Set of four meetings about room furnishing • Recorded in IDIAP SmartMeeting Room • Transcribed manually (see poster in session 3) • Annotated manually • Dialogue acts, topic segmentation, keyword assignment, argument structure, meta-events

  16. Evaluation • Wizard of Oz (WOz) methodology • Testing of existing design theories • Limited implementation sufficient • Flexibility to accommodate unexpected user actions • Comparison with other browsers (i.e. Ferret) using the Browser Evaluation Test (BET) (Wellner & Flynn 03)

  17. Future Work • Implementation of the design • Dialogue models (especially multimodality) • Layout of elements and interactive layouts • Graphics for the interface • Development of WOz environment • Initial round of testing/evaluation

More Related