1 / 14

The FAME Project

The FAME Project. Acronym: Facilitating Agent for Multicultural Exchange Partners: Universität Karlsruhe, INPG Grenoble, UJF Grenoble, ITC-irst Trento, UPC Bacelona, SONY Europe, Stuttgart ATLAS Barcelona Project volume: 5.5 M Euro

blaze
Download Presentation

The FAME Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The FAME Project • Acronym: Facilitating Agent for Multicultural Exchange • Partners: • Universität Karlsruhe, INPG Grenoble, UJF Grenoble, ITC-irst Trento, UPC Bacelona, SONY Europe, Stuttgart ATLAS Barcelona • Project volume: 5.5 M Euro • Duration: 40 months, started October 2001 • More info: http://www.fame-project.org Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  2. The FAME Projekt Facilitating Agent for Multicultural Exchange • Volume: • Duration: 40 months (since October 2001) • Financial Volume:  5,5 Mio. € • currently approx. 30 scientists • Partners: • Uni Karlsruhe , INPG Grenoble , UJF Grenoble,UPC Barcelona , ITC-irst Trento , Stuttgart, Barcelona www.fame-project.org Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  3. Project Goals • long term vision: facilitate communication between humans • reduce the workload on the users of technical equipment • observe humans and their activities in an intelligent room and serve as a context-aware information butler • FAME project goal: provide and integrate core technologies (video and speech perception, augmented reality, translation, information retrieval) to show feasibility of the concept • demonstrate system at fair • scenario 1 (lecture scenario): one person is giving a talk or lecture or presentation • scenario 2 (meeting scenario): several people are discussing / working on a common task Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  4. The FAME Showcases scenario 1 (presentation) scenario 2 (meeting) • use A/V equipment • intelligent cameraman • presentation tracking • summarisation + archiving • translation, crosslingual IR • augmented reality • video-based activity tracking • topic spotting • information butler • service: planning of fair visit Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  5. The FAME Demonstrator(at Barcelona Fair „Forum of Cultures“ 2004) FAME outside view reception by FAME-guy room gives information about spotted topics meeting inside people mention topics Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  6. The FAME Demonstrator(at Barcelona Fair „Forum of Cultures“ 2004) at the phicon wall gestures the projection table borrow a camera for photographs of the visit multimodal input on table output also on the wall interactive visit planning Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  7. The FAME Demonstrator(at Barcelona Fair „Forum of Cultures“ 2004) Back from the visit in the FAME room: dowload ... ... and look at photos select, print, save photos using phicon interaction intelligent cameraman, presentation tracker take home photos and information about FAME record testimony Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  8. Important Components • multimodal environment • context-aware intelligent camera-man automatically track people and their activities • augmented reality environment move physical icons (phicons) on table/wall, and interact with projection on table/wall • spontaneous speech recognition (with distant microphones) • translation and crosslingual information retrieval in European-English, Catalan, and Spanish • dialog and context model Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  9. Multimodal Environment at UKA Livingroomg Microphon-Array (Speaker Lokalization) Smartboard as Projection Wall Audio Signals IR-Remote Control X-10 Illumination Loudspeakers Microphones Several Beamers TV/Video 4 Cameras Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  10. Augmented Reality Table • project virtual reality on real table • move around physical icons (multiple users) • interact with projection • select, move, rotate, resize, delete, change color • write on table, pass notes to others, point to items Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  11. Intelligent Camera Man • follow speaker while talking and moving around • detect interaction from audience • zoom on area of interest e.g. when pointing somewhere or showing something Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  12. Lecture Supporter • track lecture or presentation • operate FAME room equipment by speech commands • automatically switch slides during presentation • automatically create transcript of lecture • create summary, translate to other languages • record and store all lectures in searchable database • retrieve and browse through previously recorded lectures Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  13. TO THE RECOGNITION OF THE CONTINUOUS SPEECH RECOGNITION IN NOISY AND PATTERN RECOGNITION NOT IN SECURING DUE RECOGNITION AND RESPECT Adaptation Overview estimate trigramm- model P(wn|wn-1,wn-2) tf-idf wichtige Wörter wichtige Wörter most frequent 40k words least frequent 20k words wichtige Wörter important words 72 classes P(Cn|wn-1,wn-2) · P(wn|Cn) 60k vocabulary HUB-4 corpus presentation- slides add to classes 100 Links 100 Links 20% fewer errors 100 links ±2 contexts scores CLASS 32 CLASS 14 CLASS 57 CLASS 6 CLASS 70 top n perplexity Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

  14. Welcome in Barcelona in Summer 2004 Seminar „Multimodale Räume“ Uni Karlsruhe, 14.5.2003

More Related