1 / 11

Bernd Bohnet Innovative Language Technology - TTI GmbH and University of Stuttgart

IS-FBN, IS-FBS, IS-IAC: The Adaptation of Two Classic Algorithms for the Generation of Referring Expressions in order to Produce Expressions like Humans Do. Bernd Bohnet Innovative Language Technology - TTI GmbH and University of Stuttgart Institute for Visualisation and Interactive Systems.

komala
Download Presentation

Bernd Bohnet Innovative Language Technology - TTI GmbH and University of Stuttgart

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IS-FBN, IS-FBS, IS-IAC: The Adaptation of Two Classic Algorithms for the Generation of Referring Expressions in order to Produce Expressions like Humans Do Bernd Bohnet Innovative Language Technology - TTI GmbH and University of Stuttgart Institute for Visualisation and Interactive Systems

  2. Overview of the Talk • The Framework • IS-FBN: Full Brevity extended by nearest neighbor • Properties • IS-IAC: Incremental Algorithm extended by C4.5 • Example Decision Tree • Properties • Influence of the Amount of Training Data on the Results • Summary and Conclusion

  3. The Adapted Algorithms • Full Brevity (Dale, 1989)Computes all combinations of the properties P with increasing length and therefore finds the shortest RE • Incremental Algorithm (Reiter and Dale 1992; Dale and Reiter, 1995)Psycholinguistically motivated algorithm which uses a preference ordering

  4. Description of the Method • We reused an implementation of (Bohnet and Dale, 2005)which was developed 2004 at the Macquarie University, Sydney • The implementation is based on the Paradigm: Problem Solving by Search • This is a standard AI approach which goes back to (Simon and Newell, 1963) • This approach allows to recast nearly all GRE algorithmsand to compare them in one implementation framework

  5. entity in question compare compare IS-FBN: Full Brevity extended by Nearest Neighbour • We used the FB algorithm as basis to create larger and larger RE • We used the DICE-metric to classify each of the produced RE due to their human likeness • We compared the generated referring expressions only with the expression for expressions of the same track (DIST 0.66) (DIST 0.5) (DIST 1.00) Best RE: Human REs Current RE: DIST: 0,66 0,66 0,5 1,00

  6. Properties of IS-FBN • Design goal: • get a high score • Properties: • FBN needs information for a distinct scene (track) • FBN imitates human behavior (only) • Brute force approach • It has a high DICE score

  7. entity in question present? y n Entity color (1) (2) :n :n :n Decision tree (incomplete) IAC:Incremental Algorithm extended by C4.5 • We use the IA as basis • We use C4.5 learning technique to build a decision tree • The result of a classification is the next property for IA • The input for a classification are (1) the properties and values of the entity in question (2) the already used properties Simplifying a bit the case Current RE: Æ Instance: :y :y

  8. yes yes no no Decision Tree for Furniture as build by C4.5 ‘type’ present? • The tree answers the question which attribute should be chosen next? • The tree is surprisingly small! • The read part is never chosen ‘colour’ present? ‘type’ (122/45) value of type value x-dimension sofa yes no chair fan desk size (3/0) orientation (6/3) orientation (18/11) size (5/2) value of y-dimension colour (75/12) yes no colour (6/1) Y-dimension (6/0)

  9. Properties of IS-IAC • Design goal: • “human like generation of referring expressions” • beat IS-FBN • Properties: • The result is already good with small training set • Computationally cheap • It has a high DICE score

  10. Influence of the Amount of Training Data • The following figures have been obtained by n-fold cross validation leaving one out • The values are the average for people and furniture sets • The n-fold cross validation is more precise compared to evaluation with development data: • The IS-IAC is much better with smaller training sets • The IS-FBN seems to win against the IS-IAC only with “paper thin” lead of one more training example per track! (or using the development data with a distinct distribution) • There was the possibility to use the development data + training data for the final run.Was this a possibility to “cheat” a bit? (We use the training data (only) for the final test.)

  11. Summary and Conclusion • FBN imitates human RE • IAC works more human like, but produces less human like REcompared to FBN • IAC produce already good result with small training sets • IAC does not need training data for a distinct track! • It lost unlucky against the IS-FBN only with a small lead • It was interesting and fun! • Many thanks to the Organizers and Evaluators!

More Related