Neurocognitive approach to natural language understanding and creativity

1. Neurocognitive approach to natural language understanding and creativity Włodzisław Duch Department of Informatics, Nicolaus Copernicus University, Toruń, Poland Google: W. Duch AKRR’08, Porvoo

2. Neurocognitive informatics. Intuition. Most mysterious thing about the mind … Creativity research: from psychology to neuroscience. Words in the brain: creation of novel words. Memory and pair-wise priming. Insight. Neurocognitive approach to natural language. Creation of ideas, mental models. What can we do? Plan

3. Neurocognitive informatics Computational Intelligence. An International Journal (1984) + 10 other journals with “Computational Intelligence”, D. Poole, A. Mackworth R. Goebel, Computational Intelligence - A Logical Approach. (OUP 1998), GOFAI book, logic and reasoning. • CI: lower cognitive functions, perception, signal analysis, action control, sensorimotor behavior. • AI: higher cognitive functions, thinking, reasoning, planning etc. • Neurocognitive informatics:brain processes can be a great inspiration for AI algorithms, if we could only understand them …. What are the neurons doing? Perceptrons, basic units in multilayer perceptron networks, use threshold logic – NN inspirations. What are the networks doing? Specific transformations, memory, estimation of similarity. How do higher cognitive functions map to the brain activity? Neurocognitive informatics = abstractions of this process .

4. Intuition Intuition is a concept difficult to grasp, but commonly believed to play important role in business and other decision making; „knowing without being able to explain how we know”. Sinclair Ashkanasy (2005): intuition is a „non-sequentialinformation-processing mode, which comprises both cognitive and affective elements and results in direct knowing without any use of conscious reasoning”. First tests of intuition were introduced by Wescott (1961), now 3 tests are used, Rational-Experiential Inventory (REI), Myers-Briggs Type Inventory (MBTI)and Accumulated Clues Task (ACT). Different intuition measures are not correlated, showing problems in constructing theoretical concept of intuition. Significant correlations were found between REI intuition scale and some measures of creativity. Intuition in chess has been studied in details (Newell, Simon 1975). Intuition may result from implicit learning of complex similarity-based evaluation that are difficult to express in symbolic (logical) way.

5. Intuitive thinking Learning from partial observations: Ohm’s law V=I×R; Kirhoff’s V=V1+V2. Geometric representation of facts: + increasing, 0 constant, - decreasing. True (I-,V-,R0), (I+,V+,R0),false (I+,V-,R0). 5 laws: 3 Ohm’s 2 Kirhoff’s laws. All laws A=B+C, A=B×C , A-1=B-1+C-1, have identical geometric interpretation! 13 true, 14 false facts; simple P-space, but complex neurodynamics. Question in qualitative physics (PDP book): if R2increases, R1and Vtare constant, what will happen with current and V1, V2 ?

6. Intuitive reasoning 5 laws are simultaneously fulfilled, all have the same representation: Question: If R2=+, R1=0and V =0, what can be said about I, V1, V2? Find missing value giving F(V=0, R, I,V1, V2, R1=0, R2=+) >0 Assume that one of the variable takes value X = +, is it possible? Not if F(V=0, R, I,V1, V2, R1=0, R2=+) =0, i.e. one law is not fulfilled. If nothing is known 111 consistent combinations out of 2187 (5%) exist. Intuitive reasoning, no manipulation of symbols; heuristics: select variable giving unique answer. Soft constraints or semi-quantitative => small |F(X)| values.

7. Mysterious mind … Intuition is relatively easy … what features of our brain/minds are most mysterious? Consciousness? Imagination? Emotions, feelings? Thinking? Masao Ito (director of RIKEN, neuroscientist) answered: creativity. • MIT Encyclopedia of Cognitive Sciences (2001) has 1100 pages. • 6 chapters about logics over 100 references to logics in the index. • Creativity: 1 page (+1 page about „creative person”). • Intuition: 0, not even mentioned in the index. • In everyday life we use intuition and creativity more often than logics. • The subject is getting popular … • Kenneth M. Heilman, Creativity and the Brain, Psychology Press 2005 • Mario Tokoro Ken Mogi (Sony Labs), Creativity and the Brain, 2007. • Duch W, Creativity and the Brain, W: A Handbook of Creativity for Teachers. Ed. Ai-Girl Tan, Singapore: World Scientific 2007, pp. 507-530

8. How to define creativity? Bink Marsh (2001): the number of definitions of „creativity” is equal to the number of researchers that study this subject. Sternberg (ed. Handbook of Human Creativity, 1998):„the capacity to create a solution that is both novel and appropriate”, not only in creation of novel theories or inventions, but also in our everyday actions, language understanding, interactions. Encyclopedia of creativity (Elsevier, 2005), eds. M. Runco S. Pritzke, 167 articles, but no testable models of creativity have been proposed. Journals: Creativity Research Journal, from 1988, LEA.Journal of Creative Behavior, from 1967, Creative Education Foundation. Many connections with research in: general intelligence, IQ tests, genius, special gifts, idiot savant syndrome and psychopathologies, intuition, insight (Eureka or Aha!), discovery ...

9. Psychology of creativity G. Wallas, The art of thought (1926): four-stage Gestalt model of problem solving. 4 stages: preparation, incubation, illumination and verification. Stages identified in creative problem solving by individuals and small groups of people; additional stages may be added: finding or noticing a problem, proposing interesting questions, frustration period preceding illumination, communication following verification etc. Understanding details of such stages and sequences yielding creative productions is a central issue for creativity research, but is it sufficient? Poincare (1948):math intuition and creativity is a discrimination between promising and useless ideas and their combinations; math thinking may be based on heuristic search among sufficiently rich representations. Math intuition is an interplay between spatial imagination, abstraction and approximate reasoning, and analytical reasoning or visual-spatial and linguistic thinking. This is observed in fMRI imaging (S. Dehaene, 1999).

10. Creativity with words The simplest testable model of creativity: • create interesting novel words that capture some features of products; • understand new words that cannot be found in the dictionary; • related the model to neuroimaging data. Model inspired by the putative brain processes when new words are being invented starting from some keywords priming auditory cortex. Phonemes (allophones) are resonances, ordered activation of phonemes will activate both known words as well as their combinations; context + inhibition in the winner-takes-most leaves only a few candidate words. Creativity = network+imagination (fluctuations)+filtering (competition) Imagination: chains of phonemes activate both word and non-word representations, depending on the strength of the synaptic connections. Filtering: based on associations, emotions, phonological/semantic density.

11. Symbols in the brain Organization of the word recognition circuits in the left temporal lobehas been elucidated using fMRI experiments (Cohen et al. 2004). How do words that we hear, see or are thinking of, activate the brain? Seeing words: orthography, phonology, articulation, semantics. Lateral inferotemporal multimodal area (LIMA) reacts to auditory visual stimulation, has cross-modal phonemic and lexical links. Adjacent visual word form area (VWFA) in the left occipitotemporalsulcus is unimodal. Likely: homolog of the VWFA in the auditory stream, the auditory word form area, located in the left anterior superior temporal sulcus. Large variability in location of these regions in individual brains. Left hemisphere: precise representations of symbols, including phonological components; right hemisphere? Sees clusters of concepts.

12. Words in the brain Psycholinguistic experiments show that most likely categorical, phonological representations are used, not the acoustic input. Acoustic signal => phoneme => words => semantic concepts. Phonological processing precedes semantic by 90 ms (from N200 ERPs). F. Pulvermuller (2003) The Neuroscience of Language. On Brain Circuits of Words and Serial Order. Cambridge University Press. Action-perception networks inferred from ERP and fMRI Phonological neighborhood density = the number of words that are similar in sound to a target word. Similar = similar pattern of brain activations. Semantic neighborhood density = the number of words that are similar in meaning to a target word.

13. Semantic => vector reps Word win the context: (w,Cont), distribution of brain activations. States (w,Cont) lexicographical meanings: clusterize (w,Cont) for all contexts, define prototypes (wk,Cont) for different meanings wk. Simplification: use spreading activation in semantic networks to define . How does the activation flow? Try this algorithm on collection of texts: • Perform text pre-processing steps: stemming, stop-list, spell-checking ... • Use MetaMap with a very restrictive settings to discover concepts, avoiding highly ambiguous results when mapping text to UMLS ontology. • Use UMLS relations to create first-order cosets (terms + all new terms from included relations); add only those types of relations that lead to improvement of classification results. • Reduce dimensionality of the first-order coset space, leave all original features; use feature ranking method for this reduction. • Repeat last two steps iteratively to create second- and higher-order enhanced spaces, first expanding, then shrinking the space. • Create Xvectors representing concepts.

14. Neuroimaging words Predicting Human Brain Activity Associated with the Meanings of Nouns," T. M. Mitchell et al, Science, 320, 1191, May 30, 2008 • Clear differences between fMRI brain activity when people read and think about different nouns. • Reading words and seeing the drawing invokes similar brain activations, presumably reflecting semantics of concepts. • Although individual variance is significant similar activations are found in brains of different people, a classifier may still be trained on pooled data. • Model trained on ~10 fMRI scans + very large corpus (1012) predicts brain activity for over 100 nouns for which fMRI has been done. Overlaps between activation of the brain for different words may serve as expansion coefficients for word-activation basis set.

15. Brain-like computing • I can see, hear and feel only my brain states! Ex: change blindness. • Cognitive processes operate on highly processed sensory data. • Redness, sweetness, itching, pain ... are all physical states of brain tissue. Brain states are physical, spatio-temporal states of neural tissue. In contrast to computer registers, brain states are dynamical, and thus contain in themselves many associations, relations. Inner world is real! Mind is based on relations of brain’s states. Computers and robots do not have an equivalent of such WM.

16. Memory creativity Creative brains accept more incoming stimuli from the surrounding environment (Carson 2003), with low levels of latent inhibition responsible for filtering stimuli that were irrelevant in the past. “Zen mind, beginners mind” (S. Suzuki) – learn to avoid habituation! Complex representation of objects and situations kept in creative minds. Pair-wise word association technique may be used to probe if a connection between different configurations representing concepts in the brain exists. A. Gruszka, E. Nęcka, Creativity Research Journal, 2002. Word 1 Priming 0,2 s Word 2 Words may be close (easy) or distant (difficult) to connect; priming words may be helpful or neutral; helpful words are either semantic or phonological (hogse for horse); neutral words may be nonsensical or just not related to the presented pair. Results for groups of people who are less/highly creative are surprising …

17. Creativity associations Hypothesis: creativity depends on the associative memory, ability to connect distant concepts together. Results: creativity is correlated with greater ability to associate words susceptibility to priming, distal associations show longer latencies before decision is made. • Neutral priming is strange! • for close words and nonsensical priming words creative people do worse than less creative; in all other cases they do better. • for distant words priming always increases the ability to find association, the effect is strongest for creative people. Latency times follow this strange patterns. Conclusions of the authors: More synapticconnections => better associations => higher creativity. Results for neutral priming are puzzling.

18. Paired associations So why neutral priming for close associations and nonsensical priming words degrades results of creative people? High creativity = many connections between microcircuits; nonsensical words add noise, increasing activity between many circuits; in a densely connected network adding noise creates confusion, the time need for decision is increased because the system has to settle in specific attractor. If creativity is low and associations distant noise does not help because there are no connections, priming words contribute only to chaos. Nonsensical words increase overall activity in the intermediate configura-tions. For creative people resonance between distant microcircuits is possible: this is called stochastic resonance, observed in perception. For priming words with similar spelling and close words the activity of the second word representation is higher, always increasing the chance of connections and decreasing latency. For distant words it will not help, as intermediate configurations are not activated.

19. EEG and creativity How to increase cooperation between distant brain areas important for creativity? John H. Gruzelier (Imperial College), SAN President a-q neurofeedback produced “professionally significant performance improvements” in music and dance students. Neurofeedback and heart rate variability (HRV) biofeedback. benefited performance in different ways. Musicality of violin music students was enhanced; novice singers from London music colleges after ten sessions over two months learned significantly within and between session the EEG self-regulation of q/a ratio. The pre-post assessment involved creativity measures in improvisation, a divergent production task, and the adaptation innovation inventory. Support for associations with creativity followed improvement in creativity assessment measures of singing performance. Why? Low frequency waves = easier synchronization between distant areas; parasite oscillations decrease.

20. Words: simple model Goals: • make the simplest testable model of creativity; • create interesting novel words that capture some features of products; • understand new words that cannot be found in the dictionary. Model inspired by the putative brain processes when new words are being invented. Start from keywords priming auditory cortex. Phonemes (allophones) are resonances, ordered activation of phonemes will activate both known words as well as their combinations; context + inhibition in the winner-takes-most leaves one or a few words. Creativity = space+imagination (fluctuations) + filtering (competition) Imagination: many chains of phonemes activate in parallel both words and non-words reps, depending on the strength of synaptic connections. Filtering: associations, emotions, phonological/semantic density.

21. Problems requiring insights Given 31 dominos and a chessboard with 2 corners removed, can you cover all board with dominos? Analytical solution: try all combinations. Does not work … to many combinations to try. chess board domino n Logical, symbolic approach has little chance to create proper activations in the brain, linking new ideas: otherwise there will be too many associations, making thinking difficult. Insight <= right hemisphere, meta-level representations without phonological (symbolic) components ... counting? o m black white i d o phonological reps

22. Insights and brains Activity of the brain while solving problems that required insight and that could be solved in schematic, sequential way has been investigated. E.M. Bowden, M. Jung-Beeman, J. Fleck, J. Kounios, „New approaches to demystifying insight”.Trends in Cognitive Science2005. After solving a problem presented in a verbal way subjects indicated themselves whether they had an insight or not. An increased activity of the right hemisphere anterior superior temporal gyrus (RH-aSTG) was observed during initial solving efforts and insights. About 300 ms before insight a burst of gamma activity was observed, interpreted by the authors as „making connections across distantly related information during comprehension ... that allow them to see connections that previously eluded them”.

23. Insight interpreted What really happens? My interpretation: • LH-STG represents concepts, S=Start, F=final • understanding, solving = transition, step by step, from S to F • if no connection (transition) is found this leads to an impasse; • RH-STG ‘sees’ LH activity on meta-level, clustering concepts into abstract categories (cosets, or constrained sets); • connection between S to F is found in RH, leading to a feeling of vague understanding; • gamma burst increases the activity of LH representations for S, F and intermediate configurations; feeling of imminent solution arises; • stepwise transition between S and F is found; • finding solution is rewarded by emotions during Aha! experience; they are necessary to increase plasticity and create permanent links.

24. Creativity in dementia? • Bruce L. Miller, Craig E. Hou, Emergence of Visual Creativity in Dementia. Arch Neurol. 61, 842-844, 2004. Miller et al (UCSF) describe a series of patients with frontotemporal dementia who acquired new artistic abilities despite evidence of deterioration in the left anterior temporal lobe. Good memory is common with frontotemporal dementia (FTD). Simple copying is typically preserved, some patients with FTD develop a new interest in painting, their artistic productivity can increase despite progression of the dementia. The artwork is approached in a compulsive manner and is often realistic or surrealistic in style. Why? Is it a disinhibition effect? Negation of linguistic concepts that block visual creativity? Slow “rewiring” of the cortex? Paradoxical functional compensation? Relation to TMS & savant syndrome studies (A. Snyder, MindLab Sydney).

25. Some speculations How to increase spatial coherence in the brain? Neurofeedback, or even simpler, “mantra” meditation. Simplifies neurodynamics, stops many weaker processes that pop-up. Role of neurotransmiters in creativity? Creative people store extensive specialized knowledge in temporoparietal cortex, but may switch to divergent thinking, distant associations typical for parietal system, by modulation of the frontal lobe - locus coeruleus (norepinephrine) system. Frontal lobes are involved in working memory, divergent thinking, control of the locus coeruleus-norepinephrine system. Low levels of norepinephrine => increase synchrony, large distributed activations across brain areas, creation of novel concepts. High levels of norepinephrine (mostly from locus coeruleus), more precise memory recall, localized activations.

26. Computational creativity Go to the lower level … construct words from combinations of phonemes, pay attention to morphemes, flexion etc. Creativity = space + imagination (fluctuations) + filtering (competition) Space: neural tissue providing space for infinite patterns of activations. Imagination: many chains of phonemes activate in parallel both words and non-words reps, depending on the strength of synaptic connections. Filtering: associations, emotions, phonological/semantic density. Start from keywords priming phonological representations in the auditory cortex; spread the activation to concepts that are strongly related. Use inhibition in the winner-takes-most to avoid false associations. Find fragments that are highly probable, estimate phonological probability. Combine them, search for good morphemes, estimate semantic probability.

27. Autoassociative networks Simplest networks: • binary correlation matrix, • probabilistic p(ai,bj|w) Major issue: rep. of symbols, morphemes, phonology …

28. Phonological filter • Train the autoassociative network on words from some dictionary. • Create strings of words with “phonological probability”>threshold. • Many nice Polish words … good for science-fiction poem • ardyczulać ardychstronność • ardywialiwić ardykloność • ardywializować ardywianacje • argadolić argadziancje • arganiastość arganastyczna • arganianalność arganiczna • argasknie argasknika • argaszyczny argaszynek • argażni argulachny argatywista • argumialent argumiadaćargumialenie argumialiwić • argumializować argumialność • argumowny argumofon argumował argumowalność

29. Words: experiments A real letter from a friend: I am looking for a word that would capture the following qualities: portal to new worlds of imagination and creativity, a place where visitors embark on a journey discovering their inner selves, awakening the Peter Pan within. A place where we can travel through time and space (from the origin to the future and back), so, its about time, about space, infinite possibilities. FAST!!! I need it sooooooooooooooooooooooon. creativital, creatival (creativity, portal), used in creatival.comcreativery (creativity, discovery), creativery.com (strategy+creativity)discoverity = {disc, disco, discover, verity} (discovery, creativity, verity)digventure ={dig, digital, venture, adventure} still new! imativity (imagination, creativity); infinitime (infinitive, time) infinition (infinitive, imagination), already a company nameportravel (portal, travel); sportal (space, sport, portal), taken timagination (time, imagination); timativity (time, creativity)tivery (time, discovery); trime (travel, time) Server at: http://www-users.mat.uni.torun.pl/~macias/mambo

30. More experiments • Probabilistic model, rather complex, including various linguistic peculiarities; includes priming. Search for good name for electronic book reader (Kindle?): Priming set (After some stemming): • Acquir, collect, gather , air, light, lighter, lightest, paper, pocket, portable, anyplace, anytime, anywhere, cable, detach, global, globe, go, went, gone, going, goes, goer, journey, move, moving, network, remote, road\$, roads\$, travel, wire, world, book, data, informati, knowledge, librar, memor, news, word, words, comfort, easi, easy, gentl, human, natural, personal, computer, electronic, discover, educat, learn, read, reads, reading, explor. Exclusion list (for inhibition): • aird, airin, airs, bookie, collectic, collectiv, globali, globed, papere, papering, pocketf, travelog.

31. More words Created word Word count and # domains in Google • librazone 968 1 • inforizine -- -- • librable 188 -- • bookists 216 -- • inforld 30 -- • newsests 3 -- • memorld 78 1 • goinews 31 -- • libravel 972 -- • rearnews 8 -- • booktion 49 -- • newravel 7 -- • lighbooks 1 -- + popular infooks , inforion, datnews, infonews, journics

32. Query Semantic memory Applications, eg. word games, (20Q), puzzles, creativity. Humanized interface,search + dialogue systems Store Part of speech tagger phrase extractor verification On line dictionaries Active search and dialogues with users Parser Manual

33. HIT – larger view … Learning Affective computing T-T-S synthesis Brain models Behavioralmodels Speech recognition HIT projects Cognitive Architectures Talking heads AI Robotics Cognitive science Graphics Lingu-bots A-Minds VR avatars Knowledgemodeling Info-retrieval WorkingMemory EpisodicMemory Semantic memory

34. Web/text/databases interface Text to speech NLP functions Natural input modules Talking head Behavior control Cognitive functions Control of devices Affectivefunctions Specialized agents DREAM architecture DREAM is concentrated on the cognitive functions + real time control, we plan to adopt software from the HIT project for perception, NLP, and other functions.

35. Ambitious approaches… CYC, Douglas Lenat, started in 1984. Developed by CyCorp, with 2.5 millions of assertions linking over 150.000 concepts and using thousands of micro-theories (2004). Cyc-NL is still a “potential application”, knowledge representation in frames is quite complicated and thus difficult to use. Open Mind Common Sense Project (MIT): a WWW collaboration with over 14,000 authors, who contributed 710,000 sentences; used to generate ConceptNet, very large semantic network. Other such projects: HowNet (Chinese Academy of Science), FrameNet (Berkley), various large-scale ontologies. The focus of these projects is to understand all relations in text/dialogue. NLP is hard and messy! Many people lost their hope that without deep embodiment we shall create good NLP systems. Go the brain way! How does the brain do it?

36. Realistic goals? Different applications may require different knowledge representation. Start from the simplest knowledge representation for semantic memory. Find where such representation is sufficient, understand limitations. Drawing on such semantic memory an avatar may formulate and may answer many questions that would require exponentially large number of templates in AIML or other such language. • Adding intelligence to avatars involves two major tasks: • building semantic memory model; • provide interface for natural communication. • Goal: • create 3D human head model, with speech synthesis recognition, use it to interact with Web pages local programs: a Humanized InTerface (HIT). Control HIT actions using the knowledge from its semantic memory.

37. Types of memory Neurocognitive approach to NLP: at least 4 types of memories. Long term (LTM): recognition, semantic, episodic + working memory. Input (text, speech) pre-processed using recognition memory model to correct spelling errors, expand acronyms etc. • For dialogue/text understanding episodic memory models are needed. • Working memory: an active subset of semantic/episodic memory. • All 3 LTM are coupled mutually providing context for recogniton. • Semantic memory is a permanent storage of conceptual data. • “Permanent”: data is collected throughout the whole lifetime of the system, old information is overridden/corrected by newer input. • “Conceptual”: contains semantic relations between words and uses them to create concept definitions.

38. Semantic Memory Models Endel Tulving „Episodic and Semantic Memory” 1972. Semantic memory refers to the memory of meanings and understandings. It stores concept-based, generic, context-free knowledge. Permanent container for general knowledge (facts, ideas, words etc). Hierarchical Model Collins Quillian, 1969 Semantic network Collins Loftus, 1975

39. Semantic memory Hierarchical model of semantic memory (Collins and Quillian, 1969), followed by most ontologies. Connectionist spreading activation model (Collins and Loftus, 1975), with mostly lateral connections. • Our implementation is based on connectionist model, uses relational database and object access layer API. • The database stores three types of data: • concepts, or objects being described; • keywords (features of concepts extracted from data sources); • relations between them. • IS-A relation us used to build ontology tree, serving for activation spreading, i.e. features inheritance down the ontology tree. • Types of relations (like “x IS y”, or “x CAN DO y” etc.) may be defined when input data is read from dictionaries and ontologies.

40. SM & neural distances Activations of groups of neurons presented in activation space define similarity relations in geometrical model (McClleland, McNaughton, O’Reilly, Why there are complementary learning systems, 1994).

41. Similarity between concepts Left: MDS on vectors from neural network. Right: MDS on data from psychological experiments with perceived similarity between animals. Vector and probabilistic models are approximations to this process. Sij ~ (w,Cont)|(w,Cont)

42. Creating SM The API serves as a data access layer providing logical operations between raw data and higher application layers. Data stored in the database is mapped into application objects and the API allows for retrieving specific concepts/keywords. • Two major types of data sources for semantic memory: • machine-readable structured dictionaries directly convertible into semantic memory data structures; • blocks of text, definitions of concepts from dictionaries/encyclopedias. • 3 machine-readable data sources are used: • The Suggested Upper Merged Ontology (SUMO) and the the MId-Level Ontology (MILO), over 20,000 terms and 60,000 axioms. • WordNet lexicon, more than 200,000 words-sense pairs. • ConceptNet, concise knowledgebase with 200,000 assertions.

43. Creating SM – free text WordNet hypernymic (a kind of … ) IS-A relation + Hyponym and meronym relations between synsets (converted into concept/concept relations), combined with ConceptNet relation such as: CapableOf, PropertyOf, PartOf, MadeOf ... Relations added only if in both Wordnet and Conceptnet. Free-text data: Merriam-Webster, WordNet and Tiscali. Whole word definitions are stored in SM linked to concepts. A set of most characteristic words from definitions of a given concept. For each concept definition, one set of words for each source dictionary is used, replaced with synset words, subset common to all 3 mapped back to synsets – these are most likely related to the initial concept. They were stored as a separate relation type. Articles and prepositions: removed using manually created stop-word list. Phrases were extracted using ApplePieParser + concept-phrase relations compared with concept-keyword, only phrases that matched keywords were used.

44. Semantic knowledge representation vwCRK: certainty – truth – Concept Relation Keyword Similar to RDF in semantic web. Simplest rep. for massive evaluation/association: CDV – Concept Description Vectors, forming Semantic Matrix

45. Concept Description Vectors Drastic simplification: for some applications SM is used in a more efficient way using vector-based knowledge representation. Merging all types of relations => the most general one:“x IS RELATED TO y”, defining vector (semantic) space. {Concept, relations} => Concept Description Vector, CDV. Binary vector, shows which properties are related or have sense for a given concept (not the same as context vector). Semantic memory => CDV matrix, very sparse, easy storage of large amounts of semantic data. Search engines: {keywords} => concept descriptions (Web pages). CDV enable efficient implementation of reversed queries: find a unique subsets of properties for a given concept or a class of concepts = concept higher in ontology. What are the unique features of a sparrow? Proteoglycan? Neutrino?

46. Relations • IS_A: specific features from more general objects.Inherited features with w from superior relations; v decreased by 10% + corrected during interaction with user. • Similar: defines objects which share features with each other; acquire new knowledge from similar objects through swapping of unknown features with given certainty factors. • Excludes: exchange some unknown features, but reverse the sign of w weights. • Entail: analogical to the logical implication, one feature automatically entails a few more features (connected via the entail relation). Atom of knowledge contains strength and the direction of relations between concepts and keywords coming from 3 components: • directly entered into the knowledge base; • deduced using predefined relation types from stored information; • obtained during system's interaction with the human user.

47. 20Q The goal of the 20 question game is to guess a concept that the opponent has in mind by asking appropriate questions. www.20q.net has a version that is now implemented in some toys! Based on concepts x question table T(C,Q) = usefulness of Q for C. Learns T(C,Q) values, increasing after successful games, decreasing after lost games. Guess: distance-based. SM does not assume fixed questions. Use of CDV admits only simplest form “Is it related to X?”, or “Can it be associated with X?”, where X = concept stored in the SM. Needs only to select a concept, not to build the whole question. Once the keyword has been selected it is possible to use the full power of semantic memory to analyze the type of relations and ask more sophisticated questions. How is the concept selected?

48. HIT the Web Haptek avatar as a plug-in in WWW browser. Connect to web pages, read their contents, send queries and read answers from specific fields in web forms. Access Q/A pages, like MIT Start, or Brainboost that answer reasonably to many questions. “The HAL Nursery”, “the world's first Child-Machine Nursery”, Ai Research www.a-i.com, is hosting a collection of “Virtual Children”, or HAL personalities developed by many users through conversation. HAL is using reinforcement learning techniques to acquire language, through trial and error process similar to that infants are using. A child head with child voice makes it much more interesting to play with. Haptek heads may work with many chatterbots, we focus on use of SM. Several word games with our head are here: http://diodor.eti.pg.gda.pl/

49. 20q for semantic data acquisition Play 20 questions with Avatar! http://diodor.eti.pg.gda.pl Think about animal – system tries to guess it, asking no more than 20 questions that should be answered only with Yes or No. Given answers narrows the subspace of the most probable objects. System learns from the games – obtains new knowledge from interaction with the human users. Is it vertebrate? Y Is itmammal? Y Does it have hoof? Y Is itequine? N Is itbovine? N Does it have horn? N Does it have long neck? Y I guess it is giraffe.

50. Distance calculation Euclidean distance used for binary Yes/No answer, otherwise the distance ||K–A|| is: • where |Ki–Ai| depends on the type of relation Ki and answer Ai: • - if either Ki or Ai is Unknown then |Ki–Ai|=0.5 • - if either Ki or Ai is Not Applicable then |Ki–Ai|=1 • otherwise Ki and Ai are assigned numerical values: • Yes=1, Sometimes = 2/3, Seldom = 1/3, No = 0 • CDV matrix for a single ontology reduced to animal kingdom was initially used to avoid storage size problems. • The first few steps find keywords with IG≈1. CDV vectors are too sparse, with 5-20, average 8, out of ~5000 keywords. In later stages IG is small, very few concepts eliminated. More information is needed in the semantic memory! Active dialogs.