Meaning on the Web An Empirical Design Perspective

1. Meaning on the WebAn Empirical Design Perspective • Aldo Gangemi • aldo.gangemi@cnr.it • Semantic Technology Lab • Institute for Cognitive Sciences and Technologies • National Research Council, Rome, Italy • Work described is by STLab people: • Francesco Draicchio, Alberto Musetti, Andrea Nuzzolese, Valentina Presutti • Alessandro Adamou, Eva Blomqvist, Enrico Daga

2. What is this?

3. Are these similar?

4. What do you mean? • Mathieu did it slowly and deliberately, at midnight, in the bathroom, with a knife 4

5. Killing?

6. Shaving?

7. Hanging puppets?

8. Actually ... • Mathieu buttered the toast slowly and deliberately, at midnight, in the bathroom, with a knife Donald Davidson: The Logical Form of Action Sentences, 1967 “the 'it' ... seems to refer to some entity” Davidson’s analysis started formal analysis of language in a modern way, and gave a foundation to frame logics, which were the original use case for description logics ... That ‘it’ is key to meaning 8

9. What is the key to access meaning? • When interpreting images? pattern recognition • When understanding text? parsing, resolution, expectations (background knowledge), hypothesis formation • For querying databases? key discovery • For social data mashups? application design (e.g. FB) • For linked data mashups? sparql queries, ontologies • For event recognition, situation awareness? contextontologies, cqels queries • Such “keys” provide access by putting the pieces together. Can we make ontology design as an empirical science of putting pieces of meaning together? 9

10. Cognitive background 10

11. Blink and “thin slicing” • Many organisms share the ability to gather a snap judgment about a situation in a blink of an eye (cf. M. Gladwell, Blink) • making sense of situations based on thin slices of experience: few color spots, a quick movement, a hat, the length of hair, a facial expression, the relative position of two persons, ... • Positive and negative slicing • quick reaction and expectation building vs. stereotypes and prejudices 11

12. Foreground and background • People tend to remember things because they stick out from the background (“profiling”) • but what makes a background as such? • Expectations create scenarios • even things that are not there become part of the scenario if activated by an expectation • Cf. Gestalt psychology (Köhler, Langacker, etc.) 12

13. Schema-based memory • People tend to remember items that fit into a schema. • Things that are associated through some functional similarity (cf. Gibson’s affordances) • Schemata seem to be learnt mostly inductively • blocks world, repeated verbalization of invariant scenes, peek-a-boo, etc. Cf. Deb Roy’s TED talk • Schema similar to (conceptual) frame, script, knowledge pattern, etc. 13

14. Origin of modern frames and knowledge patterns • «When one encounters a new situation (or makes a substantial change in one's view of the present problem) one selects from memory a structure called a Frame. This is a remembered framework to be adapted to fit reality by changing details as necessary … a frame is a data-structure for representing a stereotyped situation» (Minsky 1975) • Frames, schemas, scripts … «These large-scale knowledge configurations supply top-down input for a wide range of communicative and interactive tasks … the availability of global patterns of knowledge cuts down on non-determinacy enough to offset idiosyncratic bottom-up input that might otherwise be confusing» (Beaugrande, 1980)

15. How many frames? • We (STLab) are collecting, reengineering, and aligning knowledge patterns from different knowledge formats into RDF and OWL • Web data: microformats, microdata, XML patterns • Linked data: induced schemas, data patterns • (Semantic web) ontologies: ontology design patterns, extracted modules, query patterns • Linguistics: FrameNet and VerbNet frames, NLP-extracted selectional restrictions, situations from parsed NL text 15

16. Limited keys totext/discourse understanding

17. What is a window? • (i) opening in a wall • An opening, usually covered by one or more panes of clear glass, to allow light and air from outside to enter a building or vehicle. (Wiktionary) • (ii) glass-filled frame fitting an opening in a wall • Welcome to Window World, America's Largest Replacement Windows Company! (...) installs over 1 million replacement windows annually. (Ad) • (iii) glass-filled frame • A window is a transparent or translucent opening in a wall or door that allows the passage of light and, if not closed or sealed, air and sound. (Wikipedia) • (iv) glass from glass-filled frame • A pane of glass in a window. (WordNet) • (v) frame from glass-filled frame • A framework of wood or metal that contains a glass windowpane and is built into a wall or roof to admit light or air. (WordNet)

18. Genres of polysemy in WordNet • Adam’s apple (metaphor) • thyroid cartilage_1 (dul:PhysicalObject) • crape jasmine_1 (dul:Organism) • Bucket shop (diachronic metaphor) • bucket shop_2 (dul:PhysicalObject) • “(formerly) a cheap saloon selling liquor by the bucket” • bucket shop_1 (dul:Organization) • “an unethical or overly aggressive brokerage firm” • Stadium (established metonymy, “dot object”) • stadium_1 (dul:PhysicalObject) • “a large structure for open-air sports or entertainments” • arena_4 (d0:Location) • “a playing field where sports events take place” • do we need a relation between these senses, or a coarser concept? 20

19. Dot objects and co-predication (Pustejovsky, Asher) • This book is heavy but interesting • physical vs. information • Lunch was delicious but took forever • food vs. event • I saw the Coliseum in my tourist guide and wanted to go there • artifact vs. place • Actually relevant? • Power of ambiguity (“systematic polysemy”) • Minimal effort seems to count in human evolution of lexical knowledge, but only if we can easily reconstruct the context (or frame, relation, ...) • “The communicative function of ambiguity in language” (Piantadosi, Tilyb, Gibson): ambiguity allows for greater ease of processing by permitting efficient linguistic units to be re-used. “All efficient communication systems will be ambiguous, assuming that context is informative about meaning” • Also in science: inflammation has six interrelated meanings 21

20. “Fictive motion” (Talmy) • The path descended abruptly • The road runs along the coast for two hours • The fence zigzags from the plateau to the valley • The highway crawls through the city • The road leads us to Cercedilla • Need for “type coercion” to satisfy hidden frame • highway is actually a path that “can be crawled”, therefore crawling frame here is descriptive of a state, not of an action • fence is actually an object whose shape “can be followed by zigzaging” • road is actually an object that “can be followed as an indication” to our destination • sometimes an inversion of roles: the path descends because it can be descended: in general, another version of systematic polysemy 22

21. Meaning as relation • Event discovery and recognition • { Water in pot ; pot on the hot stove ; egg in water }∴ ??? • Meaning as a relational thing • Does “Egg” have a meaning independently of its relations? no, if meaning is what someone does (or might do) with something? • cf. Bartlett, Davidson, Fillmore, Minsky, Schank, Brian Smith, Gibson, ... • Dynamics (induction, abduction) of general and domain relations • Cook(Egg, Stove) as Absorb_heat(Entity, Heat_source) as Becoming(Entity, Cause) FrameNet frames have a partial order

22. Limited keys tosocial tagging analysis

23. Flickr, desire tag

24. Flickr, desire tag

25. Flickr, desire tag

26. Similarly in other folksonomies • like in FB, G+, etc. • like what? thing, comment to thing, content of cited thing, implicit negative attitude to thing? • positive/negative polarity in sentiment analysis • polarity of what? tweet, citation, irony, ... 28

27. Limited keys tolinked data analysis

28. Aggregated RDF data for “Barack Obama” (sig.ma)

29. What about the Knowledge Graph? however, where are the IRIs? named entities are only searchable within the Google engine, content is completely encrypted

30. Ontologies disagree Is that a problem? For many, yes

31. schema.org • The use cases that motivated schema.org are related to search engine requirements, probably to meet popularity and advertisement • We received schemas from the “Sponsors”. As often happens, “sponsors” create meaning, because they have either authority or money • Will ontologies be swallowed by the Sponsors? • Anyway, schemas from schema.org seem to address keys more accurately, with some exceptions ... 33

32. Pragmatic issue: we can communicate with landforms 34

33. Application issue: bone pathologies, locations, functions are all literals

34. Creating keys for the Semantic Web 38

35. Top-down: expertise patterns • Evidence that units of expertise are larger than what we have from average linked data triples, or ontology learning • Cf. cognitive scientist Dedre Gentner: “uniform relational representation is a hallmark of expertise” • We need to create expertise-oriented boundaries unifying multiple triples • “Competency questions” are used to link ontology design patterns to requirements: • Which objects take part in a certain event? • Which tasks should be executed in order to achieve a certain goal? • What’s the function of that artifact? • What norms are applicable to a certain case? • What inflammation is active in what body part with what morphology? 39

36. Layered pattern morphisms • A logical design pattern describes a formal expression that can be exemplified, morphed, instantiated, and expressed in order to solve a domain modelling problem • owl:Class:_:x rdfs:subClassOfowl:Restriction:_:y • Inflammation rdfs:subClassOf (localizedIn some BodyPart) • Colitis rdfs:subClassOf (localizedIn some Colon) • John’s_colitis isLocalizedIn John’s_colon • “John’s colon is inflammated”, “John has got colitis”, “Colitis is the inflammation of colon” expressedAs Linguistic Pattern Logical Pattern (MBox) Generic Content Pattern (TBox) Specific Content Pattern (TBox) Data Pattern (ABox) expressedAs expressedAs morphedAs exemplifiedAs instantiatedAs Logic Meaning Reference Expression Abstraction

37. Bottom-up: schema extraction (COLD2011)

38. mo:Track mo:Track mo:track mo:track mo:Record mo:MusicArtist mo:available_as foaf:maker mo:Record foaf:name dc:title mo:format mo:available_as mo:Playlist mo:Playlist Results • A method for extracting the main knowledge patterns of a LD dataset Knowledge Pattern Path Cluster of paths

39. Path Element Position 2 Path identification (length 3) (Jamendo) mo:MusicArtist mo:Signal mo:recorded_as mo:published_as rdfs:Resource rdfs:Literal mo:Record dc:title mo:Track foaf:Document mo:Playlist PREFIX mo : http://purl.org/ontology/mo/MusicArtist

40. Centrality (types) (Jamendo) mo:MusicArtist mo:Signal mo:published_as rdfs:Resource rdfs:Literal mo:Record mo:track dc:title mo:track_number mo:Track mo:available_as mo:license foaf:Document mo:Playlist PREFIX mo : http://purl.org/ontology/mo/MusicArtist

41. Centrality (properties) (Jamendo) mo:MusicArtist mo:Signal mo:recorded_as mo:published_as rdfs:Resource rdfs:Literal mo:Record dc:title mo:track_number mo:Track mo:available_as mo:license foaf:Document mo:Playlist PREFIX mo : http://purl.org/ontology/mo/MusicArtist

42. Centrality in Jamendo betweenness frequency

43. Emerging Knowledge Pattern mo:Track mo:Playlist mo:Record mo:track mo:available_as mo:MusicArtist mo:ED2K mo:available_as foaf:maker mo:available_as tags:taggedWithTag mo:image dc:date mo:Torrent dc:title dc:description tags:Tag rdfs:Literal

44. Bottom-up: Encyclopedic Knowledge Patterns (EKP) (ISWC2011) • Improving knowledge exploration and summarization by: • Empirically discovering invariances in conceptual organization of knowledge – encyclopedic knowledge patterns – from Wikipedia crowd-sourced page links • Understanding the most intuitive way of selecting relevant entities used to describe a given entity • Identifying the typical / atypical types of things that people use for describing other things • Enabling serendipitous search

45. dbpo:MusicalArtist dbpo:MusicalArtist dbpo:Organisation dbpo:Place Input data • Wikipedia page links generate 107.9M triples • Infobox-based triples are 13.6M, including data value triples (9.4M) • “Unmapped” object value triples are only 7% of page links • Paths are used to discover Encyclopedic Knowledge Patterns • Such patterns should make it emerge the most typical types of things that the Wikipedia crowd uses to describe a resource of a given type Path  Pi,j= [Si, p, Oj] linksToMusicalArtist linksToPlace linksToOrganisation

46. Encyclopedic Knowledge Patterns • An Encyclopedic Knowledge Pattern (EKP) is discovered from the paths emerging from Wikipedia page link invariances • They are represented as OWL2 ontologies

47. Paths and indicators • Emerging paths are stored in RDF according to the “Knowledge Architecture” vocabulary • Cf. our COLD2011 paper “Extracting core knowledge from linked data” • Paths and types are associated with a set of indicators

48. nRes(dbpo:MusicalArtist) dbpo:MusicalArtist Anthony_Kiedis Michael_Jackson rdf:type Jackie_Jackson rdf:type rdf:type rdf:type Chad_Smith dbpo:MusicalArtist dbpo:MusicalArtist Number of resources having type dbpo:MusicalArtist rdf:type John_Lennon Paul_McCartney rdf:type dbpo:MusicalArtist PREFIX dbpo : http://dbpedia.org/ontology/

49. nSubjectRes(Pi,j) Jackson_5 Dave_Grohl Michael_Jackson Jackie_Jackson Nirvana Oj Si dbpo:wikiPageWikiLink dbpo:MusicalArtist dbpo:Band Foo Fighters Beatles John_Lennon Paul_McCartney PREFIX dbpo : http://dbpedia.org/ontology/

50. nSubjectRes(Pi,j) Jackson_5 Dave_Grohl Michael_Jackson Jackie_Jackson Nirvana Oj Si dbpo:wikiPageWikiLink dbpo:MusicalArtist dbpo:Band Foo Fighters Beatles Number of distinct resources that participate in a path as subjects John_Lennon Paul_McCartney PREFIX dbpo : http://dbpedia.org/ontology/