1 / 6

CS315 Multimedia Search and Retrieval

CS315 Multimedia Search and Retrieval. Types of Multimedia. Multimedia: Beyond text communication Stored in a variety of formats Audio Today most popular formats: Image Today most popular formats: Animation Today most popular formats: Video Today ’ s most popular formats:.

nate
Download Presentation

CS315 Multimedia Search and Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS315Multimedia Search and Retrieval

  2. Types of Multimedia • Multimedia: Beyond text communication • Stored in a variety of formats • Audio • Today most popular formats: • Image • Today most popular formats: • Animation • Today most popular formats: • Video • Today’s most popular formats:

  3. Searching for Audio • Tagged audio files • News organizations (e.g., NPR, BBC, …) • iTunes etc services • Raw audio • Signal processing • To Speech recognition processing • To Text (inaccurate) • Perform text search • Tune recognition • Signal processing • Pattern matching • E.g., midomi.com

  4. Searching for Video • Tagged by humans • News organizations • YouTube.com • TED.com • MIT’s Lecture Browser • Lectures have additional challengesGlass at MIT Spoken Lecture Processing] Project • http://web.sls.csail.mit.edu/lectures/ • Extract audio track • Convert into text • Analyze text to reveal structure • Segment based on structure

  5. Searching for Images • Use surrounding text • Filenames, html, nearby text • Tagged by humans • Expensive! Slow! • Computer Vision techniques • Still a tough problem • Feedback based • MRS9.1 – not very successful • Tagged through games! • The ESP game (now part of gwap.com)[von Ahn, Dabbish: Labelling Images with a Computer Game] • The Peekaboom game (newer version called squigl)[von Ahn, Liu, Blum: A game for locating objects in images]

  6. Inspiration: Harness human cycles! • Captcha: program that can tell whether its user is a human or a computer • Can generate and grade test that • Most humans can pass • Current computer programs will fail • New: ReCAPTCHA

More Related