Generation of Referring Expressions: the State of the Art LOT Winter School, Tilburg 2008

1. Generation of Referring Expressions: the State of the ArtLOT Winter School, Tilburg 2008 Kees van Deemter Computing Science University of Aberdeen

2. Open Questions in GRE

3. Open Questions in GRE • Based on experiences in TUNA project • Your input is welcome! • suggestions about other open questions? • ideas about answering them

4. OQ1: Reference in context • Project proposal submitted: How can existing GRE algorithms be adapted to produce appropriate references in a (discourse or) dialogue context? • Much work exists on the choice between broad categories, e.g., pronoun vs. full NP vs demonstrative (Poesio et al; Piwek). This does not help to decide what NP to choose. Integration with GRE is needed. • Pioneering accounts are available (Krahmer & Theune 2002, Siddharthan & Copestake 2004, Stoia et al 2007), but these are tentative and largely untested. • Dialogue requires modelling of interaction between speaker and hearer (e.g., alignment and collaboration)

5. OQ2 Issues regarding knowledge and belief • How should mismatches in knowledge between Speaker and Hearer be modelled? • GRE so far has kept epistemic operators implicit: all the information in the crucial part of the KB was “shared”. • What if S and H differ?

6. OQ2 Issues regarding knowledge and belief S: [[P]]={a,b}, [[Q]]={b,c} H: [[P]]={b,c}, [[Q]]={c,d} If S says “P&Q” to refer to b, the H will misunderstand this as referring to c Shared Knowledge is not the intersection of [[P]]S and [[P]]H (or else “P” would refer to b) When S and H differ, S can initiate a clarification dialogue or try to step into H’s shoes: BelSBelH(P)? BelSBelHBelS(P)? -- Etc.

7. Relevant to the simplifications made by current GRE algorithms. E.g.: • “The present king of France” (Frege/Russell/Strawson) • “Whoever it will be, the winner of this year’s Tour de France will be less proud than last year’s winner” • It’s possible that the winner of the lottery will win 20 million. • X believes that a witch ...; Y believes that she ...”. • “The man with the martini is the murderer”, when it’s actually a soft drink (Donnellan)

8. More radical new departures needed? Consider texts about genuinely complex domains: • “We examine the problem of generating definite noun phrases that are appropriate referring expressions” (Opening sentence of the abstract of D&R 1995.) • “Bush’s Middle-East policies are a disaster. Even his closest aids have started to withdraw their support” • What do these NPs refer to? Is it realistic to want to generate them from a shared KB?

9. OQ3: Reference to sets • We know how to refer to sets making use of properties that hold of their elements. • We have gained some understanding of the role of intra-NP lexical coherence (Gatt’s thesis, Gatt & Van Deemter 2007). But, • It’s not clear how lengthy descriptions are best avoided • computational efficiency remains a difficult issue • Little is known about the use of properties that hold collectively of the referent set as a whole • Interaction between salience and sets is still a mystery.

10. OQ3: Incrementality • Studies of the TUNA corpus suggest that • incremental GRE can work very well ... • ... but only if you have a good preference order • How can good preference orders be found? • Does every new domain necessitate new empirical studies? • Or are there general principles that underlie preference orders? (E.g., frequency or complexity of a property) • Psycholinguistic issues: • Do human speakers check properties in a fixed order? • What’s the link with left-to-right order? (J.Sedivy et al. 1999)

11. OQ4: Hearer-oriented GRE • Most empirical work on reference (in the GRE community and by psycholinguists) has focussed on production. • Exceptions: Paraboni et al. (2007). Preliminary study in first STEC (Belz & Gatt 2007). • How might one build generators that optimise for the hearer? (High processing speed, low likelihood of errors) • And what if it turns out that speakers are bad at this? • If it’s practical GRE you’re interested in then this allows GRE programs to do better than human speakers. Implications for theoretical GRE are less clear.

12. OQ5 Multimodality • How does textual GRE interact with non-linguistic issues, such as • speech (e.g. pitch accent on “given” information; other prosodic issues; cf. Theune’s thesis) • pointing (e.g. Van der Sluis & Krahmer [to appear]) • salience as determined by physical proximity (as well as textual recency, intrinsic importance of objects, etc.) • facial expressions such as gaze, eyebrow movements. • These and other issues to be explored in Krahmer’s new VICI project on GRE (Tilburg, 2008-2012).

13. OQ6 Realisation & Lexical Choice • Much of what we discussed focusses on Content Determination • But referring expressions require words and syntactic constructions as well! • But surface phenomena can be difficult and interesting too • One example: Gatt’s exploration of lexical coherence • Another example: PhD project by Imtiaz Khan (Aberdeen)

14. OQ6 Realisation & Lexical Choice • Siddharthan & Copestake (2004) observed: words can introduce ambiguities. E.g. “The old president” = • the previous present, or • the president who is old (i.e., aged) • Khan: Syntax can be ambiguous as well: • “the man on the hill with the telescope” • “the old men and women”

15. OQ6 Realisation & Lexical Choice • One possible position: “avoid ambiguities at all cost”. • Khan: ambiguous strings are not only often generated, but sometimes also preferred by hearers • “the old men and women” preferred over • “the old men and the old women” • Hypothesis: surface ambiguities are balanced against other issues (e.g. brevity) • Challenge: test this hypothesis, and find an algorithm that mirrors human behaviour

16. OQ7 Reference in spacial domains • There is preliminary work (e.g. by Gatt), based on simple domains • What happens when you want to refer to an area of a country? Ross Turner’s PhD project (Aberdeen) • Input: a set of points in Scotland where ice is predicted to hamper road traffic • Example output: “icy patches are expected in the North East and on high grounds”

17. OQ7 Reference in spacial domains • Ross Turner’s PhD project (Aberdeen) • Input: a set of points in Scotland where ice is predicted to hamper road traffic. (Each point is on a road.) • Example output: “Icy patches are expected in the North East and on high grounds” • This is GRE ... but with a twist : • it may not be necessary to include all target points • it may not be necessary to exclude all other points • Referential success becomes a graded affair!

18. OQ8 Integration with the rest of NLG • GRE is arguably the most mature area of NLG: • Linguistic Realisation is the main other contender • most GRE practitioners use the same assumptions • the fact that the first NLG STEC focused on GRE confirms this • Ultimately, the GRE problem is “linguistically complete”: • if we had a flawless GRE algorithm then this algorithm could easily be transformed into an equally flawless algorithm for all of NLG ...

19. OQ8 Integration with the rest of NLG For example, • John walks [S] • (The person who) walks [ref NP] Or • A man saw a girl with earrings [S] • (The man who) saw a girl with earrings [ref NP] Or • Someone saw a beautiful girl with incredibly elaborate jade earrings bought in Paris (...) [S] • (The person who) saw a beautiful girl with incredibly elaborate jade earrings bought in Paris (...) [ref NP]

20. OQ8 Integration with the rest of NLG • Even in the absence of a flawless GRE algorithm, there is no reason why GRE should “stand alone”: • If GRE is a microcosm for all of NLG, then the rest of NLG should be able to learn from GRE – and the other way round. • Getting a better integration between GRE and other areas of NLG is a major challenge.

21. OQ9: Non-atomic knowledge • Most GRE algorithms assume that the (shared) knowledge in the input KB consists of atoms and their negations. • For example ...

22. Atomic/non-atomic knowledge • Type: furniture (abcde), desk (ab), chair (cde) • Origin: Sweden (ac), Italy (bde) • Colours: dark (ade), light (bc), brown (a) • Price: 100 (ac), 150 (bd) , 250 ({}) • Contains: wood ({}), metal ({abcde}), cotton(d) (Closed World Assumption: if an item is not listed then it’s not in the extension of the property)

23. But suppose ... • Every desk comes with a lamp. • It would be possible to add lamps {x,y} to the KB. • Additionally, we could add relations between desks and lamps to the KB, e.g. • has_lampa(y) • has_lampb(x) • This would enable us to refer to “the lamp of b” • To achieve the same effect, we could just add one general rule: • m(desk(m)  (!n(lamp(n) & has_desk(m,n)))

24. Atomic/non-atomic knowledge • More complex examples are easy to think of. • For example, perhaps we know • m(desk(m)  (n(lamp(n) &has_desk(m,n))) • lamp(x) price(x)>50 It follows that desk a cannot have 2 lamps(since 2 lamps cost more than a’s 100 pounds). Therefore • Desk a has exactly one lamp. Therefore • We can refer (uniquely) to a’s lamp. • “a’s lamp”, “the lamp” (bridging)

25. Atomic/non-atomic knowledge • To anyone familiar with KR, this will be obvious: • KR with positive & negative literals only is limited. • Even PROLOG, for example, has rules, e.g. grandfather(X Y) :- father(X Z), parent(Z Y) • Someone needs to add rules to GRE! (First attempt using Conceptual Graphs : Croitoru & Van Deemter, IJCAI 2007. There is much room for elaboration and improvement!)

26. OQ10: Integration between GRE and other areas of linguistics • Integration with psycholinguistics: • (NLG more generally: G.Kempen et al, A.Roelofs et al. Recent book by M.Guhe.) • GRE: modest beginnings in Dale & Reiter 1995 (inspiration from Levelt’s book) • investigations into optimal preference orders (Gatt et al 2007, Van der Sluis et al 2007) • Coherent reference to sets (Gatt’s thesis, 2007) • Krahmer’s VICI project aims to take this issue very seriously

27. OQ10: Integration between GRE and other areas of linguistics • Integration with syntax has so far been meagre • interleaving of Linguistic Realisation and Content Determination (Stone & Webber 1998; Krahmer & Theune 2002) • c-command constraints on the use of reflexives and pronouns (J.Odijk, DYD/ GOALGETTER systems)

28. OQ10: Integration between GRE and other areas of linguistics • Integration with formal semantics and pragmatics has been limited • Various attempts by Stone and colleagues. E.g. DeVault & Stone 2004 on vagueness (based on Kyburg & Morreau 2000) • Use of salience (mainly for category choice; see also Krahmer & Theune 2002) • GRE uses simple extensional semantics, whereas formal semantics focusses on intensionality and quantification

29. OQ10: Integration between GRE and other areas of linguistics • It would be interesting to let GRE explore core areas of formal semantics, e.g. • Use a flat KB as input (just like in GRE), to generate quantified NPs like • “Five rats died”, • “A few rats died”, • “Not all rats died”. • Find principles for choosing the quantifier pattern that’s most appropriate in the utterance situation • Early attempts by N.Creaney (2002), but limited progress so far.

30. Q11: Problematic referents • “the water in this pond” • “water” • “5”, “2+3” • “virtue”, “power” (cf. G.Chierchia’s invited address)

31. Plenty of challenges for enthusiastic young researchers!