Authoring an ontology of place semantics using volunteered geographic information

1. Authoring an ontology of place semantics using volunteered geographic information Alistair Edwardes and Ross Purves Department of Geography University of Zurich

2. Overview • Motivation for considering place • Why this is useful in the context of image retrieval • Where can we find place descriptions • How might we build semantic resources from these

3. Motivation • GI bias towards spatial representations of Geography • BUT • Not all geographic information is spatial

4. Motivation

5. Motivation • GI bias towards spatial representations of Geography • BUT • Not all geographic information is spatial • Doesn’t reflect how people experience, remember and talk about geography • What else is there?

6. Place • Space-Place Continuum • Objective-Subjective • Universal-Personal • Machine-Human

7. Where is place important? • Location-Based Services • Movement towards technologies closer to everyday, direct experience, activity based. “The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004) • Web 2.0 • Social interaction, user generated information, personal memories

8. Where is place important? [Baostar] documents place in a way that embodies neogeography, where human perspective and social interaction supercede latitude and longitude.

9. Where is place important? • Location-Based Services • Movement towards technologies closer to everyday, direct experience, activity based. “The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004) • Web 2.0 • Social interaction, user generated information, personal memories • Geographic Information Retrieval • Vernacular geography, organising activities • Photographs “GI theory articulates the idea of absolute Euclidean spaces quite well, but the socially-produced and continuously changing notion of place has to date proved elusive to digital description except, perhaps, through photography and film.” (Fisher and Unwin, 2006)

10. Place and photographs • Observer/Viewpoint • Different from universal perspective of maps • Information is perceptual • Closer to direct experience • Pre-cognitive • Many ways to interpret • Highly ephemeral • moment in time

11. Problems • Many such moments in space and time • How do we sort through them? • Image Retrieval • How do we access a description of the contents of an image? • What do we describe about an image?

12. Smeulders et al, 2000 Image Retrieval Approaches (CBIR) • Content Based Image Retrieval • “Natural” for format • Use primitive features like colour, shape and texture

13. The Semantic Gap “The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation.” (Smeulders et al, 2000) • Concept based image retrieval • Define high-level semantic concepts • Defined in loosely structured word lists (LSCOM) • Detect using low-level feature vectors

14. Image Retrieval Approaches (TBIR) • Text-based Image Retrieval • Describe contents in text • How do you access this description? • What should be described?

15. How do you access a description? • Manual annotate • Expensive and time consuming • Definitively won’t scale • Need to automate • Inconsistency amongst annotators (Markey, 1984) • Inter-annotator agreement (e.g. Ahn and Dabbish, 2004) • Controlled Vocabulary • Getty Images 12,000 keywords with 45,000 synonyms (Bjarnestam, 1998) • Specialist knowledge

16. Tripod Approach • Describe location instead • Spatial data • Geographic knowledge • Web resources

17. Obligatory Project Slide • European Commission Sixth Framework Programme Project • 3 years (started January 2007) • €3,150,000 • Partners • University of Sheffield, United Kingdom • University of Zurich, Switzerland • Dublin City University, Ireland • Otto-Friedrich-Universität, Bamberg, Germany • Cardiff University, United Kingdom • Ordnance Survey, United Kingdom • Centrica, Italy • Geodan, The Netherlands • Fratelli Alinari Istituto Edizioni Artistiche, Italy • Tilde, Latvia • Focus on image retrieval by users of professional stock photo libraries • Focus on particularly geographic images • e.g. Natural landscapes

18. What should be described? Panofsky-Shatford facet matrix – Shatford (1986)

19. SpecificOf: [Da Vinci Chambord staircase] What should be described? GenericOf: Engineering About: Innovation, technical brilliance, complexity

20. What should be described? • Advertising • Mystery • Isolation • Chocolate

21. Dimensions of Place • Theoretical Dimensions of Place • Physical setting, activities, meanings Relph (1976) • location, material form, investment in meaning Gieryn (2000) • location (spatial distribution activities), locale (the setting), sense of place Agnew (1987) • Similar to Shatford • SpecificOf – Location/Identity • GenericOf – Setting • About – Sense of place, meanings, activities

22. Organisation in Tripod • Concept Ontology • GenericOf • Scene types • Elements • About • Sense of Place • Affective, Cognitive, Conative • Qualities and Activities • Toponym ontology • SpecificOf • Identity, location

23. How can we elicit place descriptions? • Inductively • Ask people • Adjective Check List • Category Norms / Basic Levels • e.g. Mountains, Parks, Beaches, Cities • Attributes, Activities, Parts • Pick terms from a dictionary and validate • Code unstructured domain knowledge • Data mine web resources • Deductively • Look at structured semantic resources • Use a combination of these approaches

24. Free description Controlled vocabulary Sort and describe Empirical Elicitation • Online interactive experiments • Database of 150 landscape photographs from Switzerland, Germany, Holland, Italy, Portugal and the UK.

25. Overall ∩ Elements landscape village valley countryside wall field top sunlight hills cliffs castle peak fortress narrow birch town plain land hillside lawn farm creek orchard harbour fall rocks fields flowers buildings ridge bushes woods track stone ruins meadows undergrowth dunes deciduous cross coast brush wooden weather walls villa monument lane city waterfall vineyard valleys tundra trail terraced temple sunrise station ship shelter rolling riverbank pylons peninsula pathway park outcrop munros loch lightly historical hilltop ground gardens foot flood dock cycle croft covered catwalk cargo canopy canal burned brook boulder bay arm angler altitude acre Overall ∩ Qualities mediterranean summer quiet winter hilly calm isolated rural steep open mediteranian cold hot blurry windy free spring remote lonely clean dangerous cool tranquil arid well rugged rough picturesque dark vista urban snowy sandy rotting northern medieval intervention home hike fertile end dusk back Overall wooded country surrounded scene mountainous low dense coastline young world wonderland wintry waterside variation uniform two twilight tuscany tide three sunlit suburban structure streaming snowland sicily sedimented seaside seacoast savanna ruined route romantic rollign roadside riverside riverbed rising rigi region reach ranch prospect populated pond plateau photographed peeking pastures passage overgrown outback november mountaintops moor montains mist mediterran lonley lodge leafy lakeside junction heaven hanging gurgling greek france foreground following floor featureless falling dive development dawn countour copse cliif babbling area alice Overall ∩ Activities running rowing touring biking looking enjoying bycicle parking live estate

26. Subjectivity

27. How can we elicit place descriptions? • Inductively • Ask people • Adjective Check List • Category Norms / Basic Levels • e.g. Mountains, Parks, Beaches, Cities • Attributes, Activities, Parts • Pick terms from a dictionary and validate • Code unstructured domain knowledge • Data mine web resources • Deductively • Look at structured semantic resources • Use a combination of these approaches

28. Gatliff Trust Hostel on Berneray: Picture taken from the beach on Berneray of the historic Gatliff Trust Hostel. Visited in the 1990s, shortly before the causeway linking Berneray to North Uist was built. Volunteered Resources

29. www.geograph.org.uk Word lists (WordNet) Examine frequency and co-occurrence of scene types and terms with respect to a database of image captions Scene types Validation of terms

30. Scene Type Descriptions

31. How can we elicit place descriptions? • Inductively • Ask people • Adjective Check List • Category Norms / Basic Levels • e.g. Mountains, Parks, Beaches, Cities • Attributes, Activities, Parts • Pick terms from a dictionary and validate • Code unstructured domain knowledge • Data mine web resources • Deductively • Look at structured semantic resources • Use a combination of these approaches

32. Data Mining • Analysed Nouns (those used >100 in captions) • Aim • Identify similar element concepts (equivalence relationships) • Analyse noun co-occurrence with the landscape adjectives of Craik • Identify related element concepts (associative relationships) • Analyse noun-noun co-occurrence • Methodology (vector space analysis) • Identify a list of nouns (inter-annotator agreement) • Form co-occurrence vectors for each noun with nouns or adjectives • Remove insignificant occurrences (tested with chi-squared p>0.01) • Filter out vectors with few occurrences (<3) • Analyse (cosine) similarity between idf-weighted co-occurrence vectors • Visualise using hierarchical clustering and multi-dimensional scaling • Throw out largest cluster (noise)

33. Headland, outcrop, coastline, knoll, shoreline, promontory, outcrops, foreshore Landforms hill, bank, slopes, hillside, slope, cliffs, banks, crag, crags, incline, descent, fall, ascent, coombe, gradient Valley(s), ravine, gorge Land cover land, forest, fields, farmland, moor, moorland, countryside, heathland, grassland, downland Gully, channel(s), cutting ditch(es), holes, pool, ford shaft Areas, block, granite, blocks, shed, boulder(s), expanse, slab(s), pieces Element Similarities

34. Water Bridge, river(s), water(s), bank, burn, stream(s), brook, footbridge, dam, ford, pool, flood, banks, drain, plain, weir, tributary, fish, source, waterfall, bed, meadow(s), levels, gorge, fen, aqueduct, sewage confluence, riverside, reservoir(s), pipe, sluice, salmon, pools, meanders, trout, floods, waterfalls, springs, anglers, channels, table, fishery, outflow, watercourse, wharfe, otter, dike, floodplain, watershed, Built Environment Church, tower, clock, nave, porch, font, aisle House, hall, stone, wall, home, manor, brick, grounds, roof, walls, door, structure, floor, parts, window(s), period, glass, mansion, lady, storey, gable(s), architect, doorway, moat, material, façade, wing, doors, bricks, materials rubble, columns, foundations, keys, wings, village, entrance, castle, pub, cottages, inn, avenue Nature Trees, edge, wood, forest, woodland, plantation, forestry, oak, beech, inclosure, birch, pine, heathland, ponies, conifers, pines, plantations, holly, oaks, lawn, conifer, pony, spruce, sitka, larch, commoners Transport railway(s), line, station, train(s), branch, viaduct, cutting, embankment, rail(s), stations, trackbed, goods, passenger(s), terminus, mainline, gauge, platform(s), locomotive, overbridge, sidings, freight, diesel view, farm, lane, hill, footpath, valley, farmland, bridleway, hedge, heath, hillside, horse, stile, walkers, copse, leaves, chalk, picnic, warren, cyclists, spinney, trails Road, way, track, junction(s), route(s), section, access, crossing, course, traffic, direction, mile, pass, roundabout, motorway, camera, links, yards, bypass, carriageway, network, lights, pedestrian(s), loop, barrier, flyover Related Elements

35. Nature barn, bench, countryside, environment, farm, fence, field, flood, fog, grass, meadow, mist, moss, mud, parks, path, pine, rain, rural, storm, weather, wiese, wood, baum, fall, forest, leaf, leaves, tree(s), woodland, woods butterfly, insect(s), insectes, landscape, landschaft, natur, nature, scenery, bloom, blossom, cherry, flora, flower, flowers, garden, gardens, orchid, plant, plants, rose, wildflowers Weather Sun, bluesky, cloud, clouds, dusk, horizon, sky, sunshine, himmel insel, meer, sonne, strand sonnenuntergang, wasser, wolken Built environment countycourthouse, county courthouse, courthouses, cityhall, capitolbuilding, texascountycourthouses, texascourthouses, court, building(s), architecture Beaches atlantic, beach, coast, ocean, pacific, pier, plage, playa, sand, sea, seagull, seaside, shore, surf, wave(s), water alley, billboard, brick(s), castle, cathedral, centre, centreville, chimney, church, clock, county, door, entrance, façade, glass, houses, interior, metro, roof schloss, shop, sign(s), stairs, steps, store, streets, suburb, subway, tower(s), town, underground, ville, wall, window(s), wires Aviation aeroplane , aircraft, airline, airplane, airport, aviation, boeing, flughafen, flugzeug, plane, air, apache, flight, helicopter Street Art cityart, graffiti, graffitiart, graffitti, grafitti, graphiti, stencil, streetart, urbanart, paint, spray, stickers, street, wall(s), mural city, cityscape, downtown, court, skyline, skyscraper, urban, capitalcity, innercity innerlondon, capitol Flickr Work with Christian Matyas of Bamberg University

36. Develop Taxonomies Actual Nouns Concept categories Taxonomy groups

37. Conclusions • Place • Facets • Importance to geographical semantics • Eliciting place • Volunteer sources • Usefulness • Potential biases • Left open • Infrastructure • Concept detection

38. Acknowledgements • We would like to gratefully acknowledge contributors to Geograph British Isles, see http://www.geograph.org.uk/credits/2007-02-24, whose work is made available under the following Creative Commons Attribution-ShareAlike 2.5 Licence (http://creativecommons.org/licenses/by-sa/2.5/). • This research reported in this paper is part of the project TRIPOD supported by the European Commission under contract 045335.