1 / 19

Distributed multimedia architecture: indexing and optimal resources utilization

Distributed multimedia architecture: indexing and optimal resources utilization. Dana CODREANU Prof. Florence SEDES. Problem | Context. Video surveillance. Personal Information. Indexing algorithms. Patient Records. Broadcast. Large scale multimedia system :  

coen
Download Presentation

Distributed multimedia architecture: indexing and optimal resources utilization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed multimedia architecture: indexing and optimal resources utilization Dana CODREANU Prof. Florence SEDES

  2. Problem | Context Video surveillance Personal Information Indexing algorithms Patient Records Broadcast Large scale multimedia system:   ! big volume of (meta)data ! indexing algorithms diversity et heterogeneity ! mobility, localization, security, privacy , emergency management Executing any algorithm with any content by any user: =>system overload =>producing non relevant metadata 2

  3. Problem | Context =>Effective and flexible solutions for: - reducing resources consumption - security, privacy, emergency management Resources consumption reduction can be achieved in two ways: - by limiting the multimedia content transfer over the network, - by employing only the most appropriate algorithms, only over the relevant content.

  4. The Problem • Principal resources consuming : indexing process • Two aspects : • 1. Architectural solutions • -Distributed or centralized management and generation of multimedia metadata; • 2.Indexation management techniques • -Algorithms executed at acquisition time and/or at user query; • -Selection of algorithms or not, according to the user query; • Distributed or centralized executionof the algorithms;

  5. Related Work These aspects are partially considered by the existing systems:

  6. Solution | Validation The solution addresses these two points by reducing as much as possible the resources consumption through: (1) a distributed architecture with an indexing technique that avoids multimedia transfer (2) a flexible mechanism for selecting the indexing algorithms to be employed on each remote server, according to the multimedia content characteristics, its acquisition context and user queries history. Validation framework : - Proposition validated in the context of ITEA2 LINDO project for the content indexing within a multimedia distributed architecture

  7. LINDO Project LINDO : Large scale distributed INDexation of multimedia Objects • Objective: Generic Architecture for the distributed multimedia content storage, indexationandretrieval • Aim: to guide the design of the distributed multimedia information systems from different application domains • Main concern: to enable reduced resource consumption and to develop a favorable context for obtaining relevant results to the user query

  8. Example Person/ car detection Color detection Speaker change Person detection Video surveillance Video surveillance Paris Madrid Speech to text Broadcast Paris “Location: Trocadero, Paris; Time: 14 July 2011; Domain: video surveillance; Query content: abandoned bag by women in red” 10

  9. LINDO architecture | Indexing and Querying Workflows The query is executed over the most pertinent metadata collections fragments further to a complex “matching process”: Matching between query localization and remote servers description, the most relevant remote servers are selected; Matching between the query and the metadata summaries corresponding to the selected remote servers => Preliminary results; If no results => matching between the algorithms descriptions and the query => relevant algorithms . In the remote servers description, it is checked if these algorithms are already executed; If not, an explicit indexation is performed and the query is executed over the obtained metadata; The new results are displayed together with the first ones. The indexing algorithms and their descriptions are stored on the central server but they are deployed on the remote servers for the distributed indexation => no multimedia content transfer

  10. Remote servers’ description: <RemoteServer id="rs1" name="Remote Server 1"> <localisation>Bus 13, Paris, France</localisation> <description>Manages content from cameras located at the exterior of bus 13 </description> <devices> <camera id="c1Paris"> <description>located in the north corner</description> </camera> </devices> <acquisitionContext><weather><period start="2011-07-14T08:07:00 " end="2011-07-14T11:33:00">cloudy</period> <period start="2011-07-14T11:34:00 »end="2011-07-14T19:14:00">sunny</period> </weather> <luminosity><period start="2011-07-14T08:07:00" end="2011-07-14T11:33:00">75</period> <period start="2011-07-14T11:34:00   end="2011-07-14T19:14:00">100</period></luminosity> </acquisitionContext> <indexingAlgorithms> <indexingAlgorithm id="ia2rs1" name="persondetection"mediaType="video"> <description>Detects persons in outdoor area and their predominant color</description> </indexingAlgorithm> </indexingAlgorithms></RemoteServer> 

  11. Example Person/ car detection Color detection Speaker change Person detection Video surveillance Video surveillance Paris Madrid Speech to text Broadcast Paris “Location: Trocadero, Paris; Time: 14 July 2011; Domain: video surveillance; Query content: abandoned bag by women in red” 14 

  12. Metadata summaries: <?xml version="1.0" encoding="UTF-8"?> <remoteServer name="RS1"> <document src="Camera1_stream"> <localisation> <period start_time="2012-05-09T11:07:00" end_time="2012-05-09T11:08:00"/> <GPSCoordinates> … </GPSCoordinates> <!-- Trocadero, Paris--> </localisation> <localisation> <period start_time="2012-05-09T11:09:00" end_time="2012-05-09T11:10:00"/> <GPSCoordinates> … </GPSCoordinates> </localisation> <object type="Person"> <property name="color">red</property> </object> </document> <document src="Camera2_stream">.... </document> </remoteServer> 

  13. Algorithms descriptions: <AlgorithmModel AlgoName="Person Detection" MediaType="Video"> <InputParameters> <InputParamFileFormat>xml</InputParamFileFormat> </InputParameters> <OutputObject Type="Metadata"> <MetadataObject> <MetadataObjectDescription>person and color</MetadataObjectDescription> </MetadataObject> </OutputObject> <ExecutionConstraints> <MMConstraints> <Weather>windy, cloudy</Weather><Luminosity min="50" max="80"/> <DataFormat>MPEG, AVI</DataFormat></MMConstraints> <PlatformConstraints><OS>Windows</OS></PlatformConstraints> </ExecutiuonConstraints></AlgorithmModel> 

  14. Implicit and Explicit Indexation | Algorithms Selection Selection process ([Codr. et al. 2011]) Context Selection procedure Query Results List Q={f1,…,fm} L = {{ai,…,aj}, {ax,…ay},…} i,j,x,y ϵ [1,..,n] Algorithms descriptions collection {a1,…,an} 

  15. Updating implicit algorithms: Considering environmental conditions Location: parking place; Implicit Algorithms: person, car , registration plate detection ; Different luminosity and weather conditions; Some “person detection” algorithms are installed having best performances in different weather and luminosity conditions; Luminosity and weather captured by sensors and stored on the RS description ; A “person detection” algorithm is running with a best performance for sunny weather and “minimum degree of luminosity of 80%”; The weather changes, storm clouds appear, it start to rain and the luminosity drops to 50%; Based on the algorithms description the FEM module will select another “person detection” algorithm;

  16. Updating implicit algorithms: Considering User Queries history Location: parking place; Implicit Algorithms: person, car , registration plate detection ; Extracted features: {(person,0), (car, 0), (registration plate,0)} During a week multiple user queries concerning “snatched bag” occurred; The weight of snatched bag feature increases; If the weight riches a threshold t, the algorithm will be included in the implicit algorithms set

  17. Example Person/ car detection Color detection Speaker change Person detection Video surveillance Video surveillance Paris Madrid Speech to text Broadcast Paris Role: security agent Context: emergency/ normal 23

  18. Security, Privacy, Emergency • Show me all audiovisual content with lost child dressed in blue in Saint Lazare train station, Paris, on Wednesday, 29 of February, between 10 a.m and 2 p.m., from the control room by a security agent • Now the query is executed on the metadata summaries on the CS • No results are found • The system checks the rights of the agent in the current situation • Normally he does not have the right to run explicit algorithms and no rights to access the audio part of the content , but his current context unlocks temporarily this restriction. • RBAC, XACML.

  19. Distributed multimedia architecture: indexing and optimal resources utilization Questions?

More Related