1 / 25

Chapter 2: Computer models of language

Chapter 2: Computer models of language. The ingredients. By now we have encountered some of the basic ideas feeding into cognitive science • move away from associationist models of learning and behavior

gerda
Download Presentation

Chapter 2: Computer models of language

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 2:Computer models of language

  2. The ingredients By now we have encountered some of the basic ideas feeding into cognitive science • move away from associationist models of learning and behavior • information theory as a tool for exploring the nature and limits of cognitive abilities • development of “boxological” accounts of how cognitive tasks can be performed • theory of computation as a model for information-processing Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  3. Putting them together: 3 case studies • Terry Winograd and SHRDLU [TODAY] • The imagery debate [MONDAY] • Marr’s theory of vision [WEDNESDAY] Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  4. Background to SHRDLU Idea of language as: hierarchically organized planned rule-governed Computational approach to information-processing Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  5. Connections Direct analogy between formal languages and natural languages + Hierarchical organization Recursion Plans Programs Rules Algorithms Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  6. Recursion Basic idea in logic and computer science recursive definition recursive procedures Recursion is a tool for understanding the relation between complex structures and the simple elements from which they are composed Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  7. Recursive definition of wffs Well-formed formula in propositional logic: contains infinitely many propositional symbols P0,P1, . . ., together with connectives ‘’, ‘’, ‘’, and ‘’ For all i, ‘Pi’ is a wff If ‘Q’ is a wff, then ‘ Q’ is a wff If ‘Q’ and ‘R’ are wffs, then ‘Q  R’ is a wff Etc Etc Generates an algorithmic procedure for checking whether a given formula is a wff Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  8. Recursion in natural language The mouse the cat bit ran away The mouse the cat the dog chased bit ran away The mouse the cat the dog the man frightened chased bit ran away Exploit the recursive rule allowing center-embedding Problematic for associationist models – because agreement between nouns and verbs depends upon what comes later in the sentence Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  9. Natural language processing Branch of AI tackling the challenge of building a machine that can speak and understand natural languages Engineering dimension – computers that can use language to interact with people CogSci dimension – understand the computational basis of human linguistic abilities Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  10. Early examples Chatterbots (e.g. ELIZA) • typically use canned reponses to simulate conversation Machine translation • wave of enthusiasm in the 1950’s • fundamental problems with ambiguity • tried to deal with them using statistical semantics ("a word is characterized by the company it keeps”) Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  11. Winograd’s SHRDLU Narrowed down the computational challenge by restricting communication to a micro-world Interaction with the micro-world introduced a semantic dimension – allowed the application of language to the world Built on a procedural model of information-processing Constructed according to basic boxological principles (with a twist) Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  12. Micro-worlds Microworlds are artificial domains where all the possible objects, properties and events are defined in advance Many (all?) of the successes in AI have taken place in microworlds (chess, data-mining, logistics planning, credit card fraud detection. . .) Microworlds narrow down the task domain – But they also make the program more general Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  13. SHRDLU’s (virtual) microworld Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  14. What SHRDLU can do Answer questions about the block-world Resolve ambiguities Carry out instructions via a (simulated) robot hand Develop plans for carrying out complex instructions 5) Incorporate new words into its vocabulary Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  15. Limitations Purely reactive  – has no goals except those introduced in the “conversation” No sensitivity to the pragmatics of the conversation Only capable of questions and declarative sentences No need for context-sensitivity Highly simplified grammar and vocabulary Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  16. Haugeland on SHRDLU SHRDLU performs so glibly only because his domain has been stripped of anything that could ever require genuine wit or understanding. Neglecting the tangled intricacies of everyday life while pursuing a theory of common sense is not like ignoring friction while pursuing the laws of motion; its like throwing the baby out with the bathwater. A round frictionless wheel is a good approximation of a real wheel because the deviations are comparatively small and theoretically localized: the blocks-world “approximates” a playroom more as a paper plane approximates a duck. (Artificial Intelligence: The Very Idea p. 190) Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  17. Is Haugeland being unfair? SHRDLU isn’t part of a theory of common sense - nor is it intended as a complete model of natural language understanding SHRDLU shows how far we can get in modeling language understanding using certain basic tools (quasi-)hierarchical architecture procedural/algorithmic semantics Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  18. (Quasi-)hierarchical architecture SHRDLU is heterarchically organized composed of 12 separate sub-systems each sub-system has a specific task to perform But there is no overall “controller” & the sub-systems do not work in sequence  cross-talk allows sub-systems to recruit outputs of other sub-systems Sub-systems are made up of sets of procedures (algorithmic routines for solving specific problems) Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  19. Overall architecture Group 1 sub-systems Syntactic analysis Group 2 sub-systems Semantic analysis Group 3 sub-systems Integrating and using information/planning responses to commands Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  20. Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  21. The SENTENCE routine • algorithmic • calls upon other procedures • not independent of semantics Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  22. Syntax and semantics Separation of syntax and semantics is a basic principle of much theoretical linguistics and cognitive science Chomskyan generative grammar is a purely syntactic theory Cognitive scientists often assume that there is a dedicated “syntactic parsing system” But Winograd’s heterarchical approach rejects the separation Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  23. “Put the blue pyramid on the block in the box” Ambiguous: Is the blue pyramid already on the block? Is the block already in the box? This ambiguity cannot be resolved purely syntactically In fact, the syntactic sentence parser cannot run until the ambiguity is resolved But the semantic analysis also requires syntactic input Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  24. Stages in parsing sentence The parser produces “the blue pyramid on the block” as candidate NP  calls semantic analysis procedure SHRDLU checks whether there is a block with a blue pyramid on it. There isn’t. So SHRDLU backtracks to the parser, which suggests “the blue pyramid” This works. So SHRDLU takes “the block in the box” as the object of the imperative “put” Calls semantic analysis procedure to check that there is a block in the box Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

  25. Take-home messages • SHRDLU avoids many of the problems of “real-life” language comprehension (due to the way the block-world is built up) • But it shows what can be done with quasi-hierarchically organized and algorithmic routines • Illustrates how we can understand a complex cognitive ability as the product of a set of computational processes Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010

More Related