160 likes | 297 Views
Workpackage 4 Image Analysis Algorithms. Kirk Martinez, Paul Lewis and Stephen Chan Intelligence, Agents and Multimedia Department of Electronics and Computer Science University of Southampton UK. Task 4.1 User requirements Analysis.
E N D
Workpackage 4Image Analysis Algorithms Kirk Martinez, Paul Lewis and Stephen Chan Intelligence, Agents and Multimedia Department of Electronics and Computer Science University of Southampton UK Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Task 4.1User requirements Analysis • First step was to identify the requirements of the users (note overlap with other workpackages) • Required Output: Collation of scenarios and functionality • Output achieved in collaboration with other participants and delivered in the form of some initial scenarios and a set of 16 goals. These are published as part of the System design document. Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Task 4.3 Recognition Algorithm Development • PM 8-24 • Aim is to develop image analysis algorithms to meet user requirements • Required outputs: Image content analysis software and report • Consider 4.1 and 4.3 together in this presentation Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Goals • G1 Matching of similar images (includes “have you got this picture”) • G2 Automatic search using synonyms • G3 Search based on features oriented to the restoration framework - uv spotmeasures - x-ray and reverse pic views of frames - craquelure classifier -search based on “butterfly” supports in the frame Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Goals Cont. • G5 Access information quickly and easily • G6 Search based on colour • G7 Query by low quality images (especially faxes) • G8 Query by sketch • G9 Query refinement • G10 Joint retrieval by content and by text • G11 Use of publishing products built on the Artiste system • G12 Detail finding Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Goals Cont. • G13 Search using multilingual vocabulary • G14 Respect installation site privacy and security policy • G15 Produce a sustainable system after the end of the project • G16 Be consistent with partners’ predefined technical constraints Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Analysis • The goals were analysed in terms of the implications for image analysis • Possible image processing (IP) approaches were identified for goals requiring IP • Six distinct groups of algorithms were identified together with the goals to which they could contribute. Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
1.Algorithms which will find similarity matches based on global histograms (colour or grey scale) between a query image or sub-image and images in the database collections. • Could contribute to goals 1,3,7 • Useful for basic image matching • May contribute to style search and classification • Potentially faster than spatial-colour matching methods • Status: Implemented at IAM Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Example of Global Colour Histogram Search Query image Best Matches Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
2. Algorithms which will find similarity matches between a query image or sub-image and images or sub-images in the database using spatial colour distributions. • Goals: 1,3,6,10,12 • Takes into account the spatial arrangement of colours • Finds similar colour patterns at similar locations • Or similar colour patterns at any location • Status: Implemented a hierarchical colour coherence vector based matcher in IAM Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Example of H-CCV Matching Query sub-image Best Matches Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
3. Algorithms which will segment image into regions of similar texture and record feature vectors representing texture for each main regions. User can then indicate a query texture either by indicating a region in a particular image or selecting a texture from a texture palette. Images in the database containing texture regions matching the query are then retrieved. • Goals: 1,3,12 • Status: Previously implemented texture extraction algorithms • Not yet implemented automatic texture segmentation Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
4.Algorithms which will match an outline query shape with similar shapes within database images. • Goals: 1,3,4,7.8,10,12 • Pre extracting all shapes of all objects in all images is impossible i.e. can not pre-index shapes! • Techniques like the Generalised Hough Transform (GHT) use evidence accumulation to find a shape in an image and are related to template matching. • They are computationally intensive • Status: Not yet implemented for this project Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
5. Algorithms which will detect and in some cases analyse specific image features • Goals: 4 • May be able to use e.g. the GHT • Will also require specially tailored algorithms • Status: Not yet implemented Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
6. Algorithms to provide basic image manipulation • Goals: All involving image handling • Include operations like image conversion, compression, scaling, rotation etc • Most are widely available but may need re-implementing or tailoring in context of Artiste • Status: Partial implementation Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000
Environment for Algorithm Testing • Test-beds in the IAM lab include VIPS and MAVIS 2 • Algorithms developed as “stand alone” modules which deliver feature vectors (FVs) and modules for calculating similarity between FVs • MAVIS 2 is a multimedia retrieval and navigation environment • It associates media content and the concepts they represent • Concept layer equivalent to a thesaurus • Allows integrated content and concept based searching with query scope expansion Project IST_1999_11978 -ARTISTE – An Integrated Art Analysis and Navigation Environment Review Meeting N.1: Paris, C2RMF, November 28, 2000