Klaus Lepsky – Thomas Müller – Jens Wille Metadata improvement for image information retrieval

1. Klaus Lepsky – Thomas Müller – Jens WilleMetadata improvement for image information retrieval

2. Problem I Rogier van der Weyden. St. Columba Altarpiece. Adoration of the Magi. Central part. c.1455. Oil on panel. Alte Pinakothek, Munich, Germany Triptych with the Adoration of the Magi, so-called Columba Altarpiece, Munich, Alte Pinakothek, inv. nr. WAF 1189-1191. Center panel: ca. 140 x 153, Wings each: ca. 140 x 73 cm. Title: St Columba Altarpiece (central panel) Author: ROGIER VAN DER WEYDEN Date: c. 1455 Technique: Oil on oak panel, 138 x 153 cm Type: religious Form: painting Location: Alte Pinakothek, Munich

3. Problem II Rogier van der Weyden Weyden, Rogier /van Rogier <van der Weyden> Pasture, Rogier /de le Rogier <de LePasture> Roger <de LePasture> Rogier <de le Pasture> Roger <de le Pasture> Regelet <de le Pasture> Rogelet <de le Pasture> Pasture, Rogier /de Weyden, Roger /van der Roger <von Brügge> Roger <de la Pasture> The prometheus database with • more than 400.000 pictures from 50 different collections • descriptions without any standardization • weak subject indexing • resulting in low recall ! Adoration of the magi St. Columba Altarpiece Triptych with the Adoration of the Magi St Columba Altarpiece (central panel) St. Columba Altarpiece. Adoration of the Magi. Central part Triptych with the Adoration of the Magi, so-called Columba Altarpiece Columba Altarpiece Adoration of the Magi. Central part

4. Idea I Art historical texts are texts about works of art and therefore may be used as ‘metadata’ for these works. The central Adoration of the Magi, the only depiction of this kind in Rogier's extant work, was a particularly popular subject in Cologne, for since the 12th century the city had kept relics of the Magi, its most treasured possessions, in the cathedral. The Netherlandish artist not only created this work for a patron in Cologne, but also drew inspiration from an outstanding painting done in Cologne at this period, Stefan Lochner's Altarpiece of the Patron Saints of Cologne, sometimes known as the Cathedral Altarpiece. This large retable, painted in the 1440s for the chapel of Cologne town council, unites the patron saints of the city in a magnificent, monumental work from which any trace of historical narrative has been eliminated. Rogier, perhaps at the special wish of his patron, adopted the almost central position of the Virgin Mary in the depiction of the Adoration; this was an unusual situation for her in Netherlandish art. He also echoed other motifs, including the figure of a woman seen in profile, dressed in green and with a plait of hair, who appeared on the left wing of Lochner's altarpiece and again in Rogier's Presentation; details like the sword and fluttering hatband of the youngest king; and, behind the Magi, the figure of a bearded man holding a hat in front of his chest. Traces of the late Romanesque church of St. Gereon in Cologne seem to have entered into the architecture of the temple on the right wing - the polygonal rotunda, the gallery, and the small buttresses on the outside. (http://www.wga.hu/frames-e.html?/html/w/weyden/rogier/11columb/0columb.html) Improving information retrieval of digital images by automatically connecting them with text based description

5. Idea II article “Drei Könige” (three kings) homepage of rdk-web

6. Idea III identifiying names and bringing together synonymous forms matching text and descriptions of pictures • identifying corresponding text by • finding names of artists in text • finding names of works of art in text • bringing together pictures and corresponding text • developing a system for an automatic standardization of metadata and a matching algorithm for metadata and text to improve poorly standardized metadata

7. lingo - linguistic approach to automatic text indexing wordsearcher: linguistic analysis, grammatical variants are derived to one basic form kings > king #n religious > religious #a, religion #n multiworder: linguistic/lexical analysis, dictionary-based identification of phrases „rogier van der weyden“ „adoration of the magi“ sequencer: algorithmic identification of phrases, based on sequences of the word classes holy kings > #a #n > „holy kings“ synonymer: identifying and generating of synonyms Pasture, Rogier /de le > „rogier van der weyden“ adoration of kings > „adoration of the magi“

8. importance of synonyms synonyms of names of pictures Adoration of Kings The Adoration of Kings The Adoration of the three holy Kings Adoration of the magi Rogier van der Weyden Weyden, Rogier /van Rogier <van der Weyden> Pasture, Rogier /de le Rogier <de LePasture> Roger <de LePasture> synonyms of names of artists

9. perseusMatch matching algorithm with levenshtein distance using word distance! (1) grammatical variants editing distance for characters('königs', 'könige') = 1 (editing distance for characters('kings', 'king') = 1) editing distance for words('königs', 'könige') = 0 (editing distance for words('kings', 'king') = 0) because basic form('königs') = basic form('könige') (basic form('kings') = basic form('king')) (2) synonyms editing distance for characters('könige','weise') = 6 (editing distance for characters('kings','magi') = 5) editing distance for words(‘könige’,’weise’)= 0 (editing distance for words(‘kings’,’magi’)= 0) because synonyms('könige') contains 'weise' (synonyms('kings') contains 'magi')

10. connecting text & pictures "[...] Nic. Lancret stellte in der vor 1732 für den Marquis de Béringhen gemalten Elementenfolge das Wasser durch eine ländliche Ff.szene dar, die auch Allegorie des Liebeswerbens ist (s. Sp. 256; Abb. 52; Georges Wildenstein, L., Paris 1924, S. 70 Kat.nr. 2, s. auch Nr. 3, Abb. 5). [...] Luna­Diana als Herrin über Fischerei und Schiffahrt zeigt ein Stich von Jan Saenredam nach Hendrick Goltzius (um 1559: Abb. 22)." lingo & perseus match KünstlerIn: Hendrik Goltzius Titel: Allegorie Standort: Basel, Öffentliche Kunstsammlung Gattung: Malerei Datierung: 1611

11. Thank You for listening!