440 likes | 592 Views
Technical University of Crete Department of Electronic and Computer Engineering. Multimedia Data Management Euripides G.M. Petrakis http://www.intelligence.tuc.gr/~petrakis http://courses.ced.tuc.gr Chania 2010. Definition.
E N D
Technical University of CreteDepartment of Electronic and Computer Engineering Multimedia Data Management Euripides G.M. Petrakis http://www.intelligence.tuc.gr/~petrakis http://courses.ced.tuc.gr Chania 2010 Introduction
Definition • Multimedia: composite entities combining text, audio, images, video (bit-stream objects), graphics • Multimedia Information Systems: database systems that support all multimedia data types and handle very large volumes of information Introduction
TEXT • The most common type of information • The least space intensive data type • The form in which text is stored varies (plain ascii, word files, spreadsheets, annotations, database fields etc.) • Text fonts are becoming complex allowing special effects (color, shade, fill etc.) Introduction
AUDIO • Space intensive (one minute can take up Mbytes), presented as analog, digital or MIDI • Analog waveform: electrical signal, • amplitude specifies the loudness of the sound • in microphones, tapes, records, amplifiers, speakers • Digital waveform audio: digital, • less sensitivity to noise and distortion • involves larger processing and storage capacities • Digital Audio Tape (DAT), Compact discs (CD) • WAV (Microsoft’s wave file format) Introduction
MIDI (Musical Instrument Digital Interface) Furht et.al.96 • Commands that describe how the music should be played are stored (instead of sound) • A music synthesizer generates sound • Provides high data compression, • Widely accepted Introduction
IMAGES • Digital images: sequences of pixels • Pixels: numbers interpreted to display intensity, color, contrast etc • Binary (0-1 values), gray-scale (8 bits/pixel), colour (3x8 values for RGB) • Space overhead depends on image type, resolution, compression scheme • Image formats: tiff, bmb, jpeg etc. Introduction
Image Concepts and Structures • Binary images: 1 bit/pixel • black & white photos, facsimile images • Computer Graphics: 4 bits/pixel • Grayscale images: 8 bits/sample • Color images: 16, 24 bits/pixel Introduction
RGB Representation • A color is produced by adding • red, green and blue • The straight line R=G=B specifies gray values ranging from black to white Introduction
YUV Representation • YUV describes the luminance and chrominance components of an image • 1 luminance: gray scale version of an image • Y = 0.299R + 0.587G + 0.114B • 2 chrominance components: • U = 0.564(B - Y) • V = 0.713(R - Y) Introduction
Conversions • Conversion between RGB and YUV requires multiplication operations • an approximation: • Y = R/4 + G/2 +B/2, U=(B-Y)/2, V=(R-Y)/2 • R = Y + 2V, G = Y – (U + V), B = Y + 2U • YCbCr is another color format • for compression • Cb = U/2 + 0.5, Cr = V/1.6 + 0.5 Introduction
VIDEO • The most space intensive data type • A sequence of frames • Realistic video playback, transmission, compression/decompression require transfer rates about 30frames/sec • Microsoft’s AVI and Apple’s Quicktime file formats integrate video and audio in the same presentation Introduction
Sequence of Frames Khoshafian Baker 96 Introduction
Audio-video Modes of Operation Furht et.al.96 • Can be either stored or used / transmitted live in real-time • Can be used interactively or non-interactively Introduction
GRAPHICS • Objects described through their basic elements (e.g., 2D, 3D shapes) • these elements can have different sizes, position, orientation, surface, fill etc. • compact representations • generated and can be manipulated by design tools (e.g., CAD tools) • Their descriptions are stored in files Introduction
2D and 3D graphics objects Khoshafian Baker 96 Introduction
MULTIMEDIA objects • Text, audio, images, video, graphics are elements of complex multimedia objects • Various tools or applications integrate, process and combine multimedia • Applications: multimedia authoring applications that output documents and databases and end-user applications (e.g., video on demand) • Tools: for viewing, updating, querying (presentation viewers, browsers etc.) Introduction
Multimedia Databases (MDB) • Means stored information or database management systems (dbms) • Multimedia dbms (mdbms) integrate conventional database capabilities together with different technologies such as Hierarchical storage management (HSM) and Information retrieval (IR) Introduction
Multimedia Technologies • Technologies integrated within a mdbms • HSM support • IR support (exact and approximate) • Spatial data types and queries • Interactive querying, relevance feedback, refining • Automatic feature extraction • Automatic content retrieval and indexing • Query optimization Introduction
Database capabilities • Persistence: object persist through invocations • Transactions: content is inserted, deleted, updated • Concurrency control: transactions run concurrently • Recovery: failed transactions are not propagated to the db • Querying: content can be retrieved • Versioning: access previous states of objects • Integrity: transactions guarantee consistency of content • Security: constraints for accessing/updating objects • Performance: optimal data structures and programs Introduction
Hierarchical Storage Management (HSM) • Support storage of multimedia objects • On-line: on RAM, magnetic disk • Near-on-line: on optical storage • Off-line: on tapes, shelves • Each level has different • Performance: decreases from top to bottom • Capacity: increases from top to bottom • Cost: decreases from top to bottom Introduction
Information Retrieval (IR) Capabilities • Retrieval is the most common operation • Deletions and updates are less common • Exact match: search based on exact information • Inexact: search based on inexact information e.g.,partial, neighborhood search, can be fuzzy or probabilistic • The results are ranked by order of relevance to the query • Query refinement • Iterate over query results • Adjust weights of query terms or features • And finally resubmit queries Introduction
MDBMS architecture Khoshafian Baker 96 Introduction
MDBMS Implementation • Relies on 3rd party vendors for each component • Relational dbms for typical records • separate optical storage module for text/audio/graphics/images/video • Text retrieval system (e.g. Lucene) • Audio/image/video retrieval system • Feature extraction system • Multimedia object interface system Introduction
Object-Oriented Multimedia Databases • Better design, better suited for multimedia applications • Uniform handling of data and operations • Data types are objects with internal structures and operations that capture the behavior of objects (e.g., audio playback, video browsing) • OO dbms does not satisfy all MM requirements • Provides primitives for object handling • Multimedia components need to be implemented or integrated Introduction
Object orientation in MMDBMS’s Khoshafian Baker 96 Introduction
Client-Server Architectures • Multimedia databases operate in client-server architectures • A number of interconnected server nodes provide I/O capabilities for multimedia objects • Basic DBMS module • IR module • HSM module • More services may include scanner, fax services etc. Introduction
Client-Server in mdbms Khoshafian Baker 96 Introduction
Various modules and servers in an MMDBMs architecture Khoshafian Baker 96 Introduction
Multimedia Applications • Multimedia Systems suggest a variety of applications • Multimedia conferencing • Multimedia on demand (interactive TV, news on demand) • See next page for more … Introduction
Multimedia Applications Furht et.al.96 Introduction
Multimedia Conferencing • Multimedia conferencing enable a number of participants to exchange multimedia information • Skype, PoWWoWNow, Webex • Each participant has a workstation linked to other workstations over high-speed networks • Each participant can send or receive mm data and perform certain collaborative activities Introduction
A video conference system Furht et.al.96 • The biggest performance challenge occurs when the participants transmit voice and video • These are mixed together to form a composite stream consisting of video and voice streams Introduction
Software Architecture Furht et.al.96 Introduction
System Functions • Multipoint connection set-up: the system negotiates for network resources • Dynamic session control: add/delete participants • Conference directory service: supports registration, announcement, query etc. • Automatic scheduling and recording: planning of network resources. • Conference teardown: release reserved resources Introduction
Architectures • Fully distributed: direct connections between the participants • Processing and mixing of media at every location • Shortest delay • The connections increase rapidly • Centralized (star) network: a central is connected to every participant • Processing and mixing at central node • The central node waits until all media is received before mixing and broadcasting Introduction
Architectures (cont.) • Double star network: a central node from one star network is connected to another central node of another star network • Hierarchical network: intermediate nodes, root and leaves (participants) connected hierarchically • intermediate nodes perform mixing and processing • the completely mixed data is sent to root who broadcasts directly to the leaves • reduces network traffic significantly Introduction
Multimedia conferencing network architectures Furht et.al.96 Introduction
Video on Demand • Fast networks coupled with powerful computers and compression techniques will be capable of delivering stream data in real-time • On-demand multimedia services • interactive entertainment • video news distribution • video rental services • digital multimedia libraries Introduction
Interactive Television (ITV) • An ITV system must be capable of providing • basic TV • subscription TV • pay per view • video on demand • shopping • education • electronic newspaper • financial transactions • single-user and multi-user games Introduction
ITV network Architectures • Distributed architectures, components • information (content) servers: entertainment companies, TV stations connected to stb’s offering different types of services • network: wide area network (ATM) connects the head-ends which are connected with subscribers • set top boxes (stb) : terminal devices that allow users interact with a network providing personalized, on-demand, interactive services Introduction
A General ITV Architecture Furht et.al.96 Introduction
Hierarchical ITV Architecture Furht et.al.96 Introduction
News on-Demand • Delivers news on demand to various corporate and financial services • The Nynex and Dow Jones systems: • The Dow Jones production center produces news which are send through a wideband network to the Nynex media service center • Customer sites can request specific news from the Media Service Center which are delivered over the network and stored at customer sites Introduction
Architecture of Nynex and Dow Jones system Furht et.al.96 Introduction