Introduction: Multimedia Databases

1. Introduction: Multimedia Databases MMDB - Introduction

2. Today’s Media Types • Discrete media: Time is not part of the semantics of the media • Classical Data • Text/Documemt • Image • Continuous media: Time or more exactly time-dependency between information items, is part of the information itself. • Video • Audio MMDB - Introduction

3. What is an MMDBMS? A multimedia database management system (MMDBMS) is a framework that manages different types of data potentially represented in a wide diversity of formats on a wide array of media sources. MMDB - Introduction

4. Abilities of MMDBMS • Query data represented in different formats. • Retrieve media objects from a storage device in a smooth jitter free manner. • Take the answer of a query and develop a presentation of that answer in terms of audiovisual media. • Deliver this presentation in a way that satisfies various quality of service requirements. MMDB - Introduction

5. Sample Multimedia Scenario Consider a police investigation of a large-scale drug operation. This investigation may generate the following types of data. • Videodata captured by surveillance cameras that record the activities taking place at various locations. • Audio data captured by legally authorized telephone wiretaps. • Image data consisting of still photographs taken by investigators. MMDB - Introduction

6. Sample Multimedia Scenario • Document data seized by the police when raiding one or more places. • Structure relational data containing background information, bank records, etc., of the suspects involved. • Geographic information systems data containing geographic data relevant to the drug investigation being conducted. MMDB - Introduction

7. Data Sources Used Surveillance video Audio data Police application Geographic information Still image data Relational data Document data MMDB - Introduction

8. Characteristic of Multimedia Queries • We normally retrieve a few records from a traditional DBMS through the specification of exact queries based on the notions of “equality”. • The types of queries expected in a multimedia DBMS are relatively vague or fuzzy, and are based on the notion of “similarity”. • The indexing structure should be able to satisfy similarity-based queries for a wide range of similarity measures. MMDB - Introduction

9. Example: Image Queries Image-based queries: • Police officer has a photograph in front of him. • He want to find the identity of the person in the picture. • Retrieve all images from the image library in which the person appearing in the photograph appears. • To support this, we need to know what “similarity” means. • We need to be able to efficiently support these operations. MMDB - Introduction

10. Example: Image Queries Keyword-based queries: • Police officer wants to examine picture of Bill Clinton. • Retrieve all images from the image library in which Bill Clinton appears. • To support this, we need to know how to associate different attributes with images (or parts of images). • We need to know how to effectively index and retrieve images based on such attributed. MMDB - Introduction

11. Example: Audio Queries Query 1: • Police officer is listening to an audio surveillance tape. • Search through a voice log database to see if the identity of the person, whose voice is on the tape, can be ascertain. Query 2: • Police officer wants to review all audio logs that Bill Clinton participated in the conversations. • Find all the audio tapes in which Bill Clinton was a participant. MMDB - Introduction

12. Example: Text/Video Queries Text query: • Find all the news that deal with president scandals. Video query: • Find all video segments in which Bill Clinton appeared with Monica Lewinsky. MMDB - Introduction

13. MMDB Research Issues: Queries • Need a single language within which multimedia data of different types can be accessed. • The language must be able to merge, manipulate, and join together, results from different media sources. • Once such language are devised, we need techniques to: • Optimize a single query • Develop servers that can optimize processing of a set of queries. MMDB - Introduction

14. MMDB Research Issues: Contents • How should we extract the content of • an image • a video-clip • a audio-clip • a free/structured text document • How should we index the results of this extracted content? • What algorithms can be used to efficiently retrieve media data on the basis of similarity? MMDB - Introduction

15. MMDB Research Issues: Storage • How is media data laid out on storage devices? • How do we design disk/CD-ROM/DVD servers so as to optimally satisfy different clients concurrently, when these clients execute the following operations • Playback • Rewind • Fast forward • Pause MMDB - Introduction

16. MMDB Research Issues: Presentations and Delivery • How can we deliver a multimedia presentation to users when there is • a need to interact with other remote servers to assemble the presentation? • a bound on the buffer, bandwidth, load, and other resources available on the system? • How can such presentations optimize Quality of Service (QoS)? MMDB - Introduction