Design of Web Agents Inspired by Brain Research

1. Design of Web Agents Inspired by Brain Research Maya Dimitrova, Hiroaki Wagatsuma * and Yoko Yamaguchi* Institute of Control and System Research – Bulgarian Academy of Sciences *Laboratory for Dynamics of Emergent Intelligence, RIKEN Brain Science Institute, Japan

2. Content 1. Introduction 2. Information retrieval studies based on cognitive heuristics (problem definition) 3. Theta phase coding in the brain and episodic memory 3.1. The Theta phase model 3.2. Relevance to episodic/autobiographical memory 4. Autobiographical memory in Web context: An example 5. Conclusions and future work ADBIS'07 Varna

3. Introduction • Modeling agents’ own experience – their reconstructed past - is essential to optimize agent behavior (i.e. agents for Web knowledge search, retrieval and presentation) • We propose to base this optimization on recent studies and models of the brain – focusing on the generality of processing in the animal and human, not on their difference • We have focused on “episodic” memory and the underlying hippocampal mechanisms of processing as closest to the “story-telling” approach of agent design ADBIS'07 Varna

4. Information retrieval studies based on cognitive heuristics • Problem statement: • Current Web page classifiers perform excessive amount of processing to reach simple classificatory results • Whenever the classificatory task is multi-dimensional, agent performance deteriorates dramatically • There is the database or storage problem – where models have always been least analogous to human memory • Human memory processing is fast, heuristical and much more efficient in the multi-dimensional case, than in the sequential processing case (contrary to the machine) etc. (recent finding) • There is always a point in looking into recent brain studies – for their fast and accelerated development as empirically-based science ADBIS'07 Varna

5. Information retrieval studies based on cognitive heuristics (2) • The current study: • Presents the main ideas of the Theta phase precession theory as proposing a memory model, different from the mainstream rate-coding models • Operationalizes “episodic memory” for the purpose of design of memory-optimized Web search agent • Provides an example to illustrate the described process in a human Web search process and some relevant conclusions for agent future development ADBIS'07 Varna

6. The agent (1) • In earlier work we designed a Java-servlet (a wrapper to a search engine) to diagnose the level of expertise and detail of a corpus of on-line available Web pages • Weak point – what to store in the database and in what form ADBIS'07 Varna

7. The agent (2) • It employed pattern classification based on syntactic analysis for diagnosing the amount of detail • And cognitive heuristics based on memory research for the expert level of the text ADBIS'07 Varna

8. The agent (3) • In following history of interaction, optimization is needed much the way natural cognitive systems perform - learn, abstract, remember and forget • We use the ‘autobiographical memory’ metaphor to apply to such kind of Web agents and the Theta phase precession theory to model it ADBIS'07 Varna

9. Current MATLAB GUI versionof the agent interface • The difference of the MatLab version from the Java version is that the Java-servlet processed also the hyperlink structure of the Web page • The current version is simplified for algorithmic and modeling reasons ADBIS'07 Varna

10. Assumptions of the current study: • Assumptions: - User interaction with the present-day Web is an autobiographical experience (remembering sequences of events like personal-life episodes) • Memorizing events in a human-like memory system undergoes constant synthesis as any other learning process and therefore can be implemented in autonomous agent on the Web ADBIS'07 Varna

11. Operational definitions in the current study: • Definition of Episodic Memory: • The system of processing episodes of Web search and transforming them into autobiographical events of successful/unsuccessful Web navigation to reach the needed goal • Every Web page viewed by the user is a complex perceptual and semantic stimulus, i.e. an episode of Web search • What is stored in episodic memory from the rapid viewing of a succession of pages is a set of fragments of the search episode rather than the entire pages and their exact order of viewing ADBIS'07 Varna

12. Hypotheses of the current study: • The theoretical hypothesis is that episodic memory extracts meaningful cues from the fragments of the page-retrieval episodes, rather than random perceptual or semantic cues • “Meaningful” we define as having anticipatory value (i.e. useful in the future) • The tested hypothesis is that the mechanisms of event synthesis from fragments of page-retrieval episodes are the same in personally-significant and neutral situations, that is – universal for learning per se • These effects are reproduced by both the human and the agent in the current study ADBIS'07 Varna

13. Theta phase coding in the brain and episodic memory • Theta phase theory models rhythmic patterns of activity of assemblies of neurons in the brain and their role in complex cognitive tasks (theta rhythm 4-10 Hz) • Theta phase modulation is associated with learning and memory in rats and humans A mechanism of pattern compression has been observed in hippocampal neuron assemblies, which is called theta phase precession • Modelling hippocampal nerve cirquit by the Theta phase precession theory (Yamaguchi, Yamaguchi et al.) reinstates complex learning effects observed in human episodic memory like object-context integration, single-trial learning and time-space contextualization ADBIS'07 Varna

14. Neuro-realistic model of episodic encoding and retrieval • (a) Main modules of the hippocampus – ECII, ECIII, CA3, CA1, DG • (b) Entorhinal input, emerging from the sensory system and representing the vector time series produced by behavior • (c) when the rat runs to the right, the phase shift in firing within each theta rhythm cycle occurs in place cells 1 to 4, which are activated sequentially ADBIS'07 Varna

15. Properties of the phase model • (The recurrent connections in the CA3 layer of the hippocampus form asymmetric connections so that units with prior activations are connected to those with subsequent activations • These activations are a consequence of the coincidence between their phase differences and the time window of the asymmetric synaptic plasticity ADBIS'07 Varna

16. Model of hippocampal nerve circuit • (a) mechanism of theta phase coding • (b) phase shift in firing within each theta rhythm cycle ADBIS'07 Varna

17. Relevance to modeling memory: • Partial cuing triggers pattern completion in the CA3 and its transmission to the CA1 and EC. Thus a stored pattern in the hippocampus can be sent to the cortical area to replay the input pattern • The temporal difference between encoding and retrieval in the network parallels behavioral tests • The retrieval is ten times faster than the behavioral input in the encoding stage and the retrieval time scale is similar to that of the compressed pattern of theta phase precession • The retrieved activity can cause further synaptic plasticity in other cortical areas for memory consolidation or memory transfer from the hippocampus to the cortex ADBIS'07 Varna

18. Importance of theta phase precession theory to the present study: • The emergence of rhythmic patterns in the absence of memory to cause its formation is a plausible candidate for a neural correlate of the formation of complex and durable cognitive effects governing future behavior i.e. autobiographical memories • We have aimed to give a relevant example to illustrate these effects ADBIS'07 Varna

19. How to find an Art Gallery on the Web with incomplete initial information? • Participants: - Motivated - Neutral • Search goal: - Example of new concept learning, similar to artificial concept learning: “Kurenai Kai” • Search task: - Google search from incomplete initial information (The name of the gallery “Kurenai Kai” not known apriory) ADBIS'07 Varna

20. Results and discussion: Table 1. User performance on Web search with incomplete initial information about the search goal • Our study of Web search has replicated the mechanism of performance of the theta phase precession model : • The Web paths resemble random search until a meaningful path is composed • We observe that meaningful paths are composed, rather than encountered ADBIS'07 Varna

21. Web search: “Kurenai Kai” • Left path represents the nature of the search with incomplete initial information • Right path represents the shortest path with well-remembered concept term ADBIS'07 Varna

22. Remembering meaningful fragments of search episodes Left path represents episodic memory guided retrieval based on a set of meaningful cues and “insight” The pages to the right represent the mappings to the respective initial cues (none of them are identical) ADBIS'07 Varna

23. Conclusions from the Web search study: • Users base subsequent retrieval of the search path on remembering the ‘events’ of their search, which are the meaningful fragments of knowledge inside the Web pages, not the exact ‘page-retrieval’ episodes • We propose that the autonomous Web agent behaves in analogy with the event-based (autobiographical) strategy that Web users are applying for retrieval of useful information, which is some state of knowledge with anticipatory value (useful in the future) • Next will be tests of the agent for optimized storage of fragments of knowledge contained in the Web page, rather than of snapshots of sequences of Web pages ADBIS'07 Varna

24. Agent reinstatement of the search task Fragment extraction and recombination of textual elements as a process of encoding search events ADBIS'07 Varna

25. Temporal evolution of episodic events by the neuro-cognitive agent ADBIS'07 Varna

26. Conclusions and Future Work • We presented one approach towards the design of Web agents based on recent knowledge from brain research • We have tried to demonstrate how the theoretical framework of theta phase coding endows real-time process of memory formation of Web search experiences and the retrieval process of effective routes to the targets through the experienced sites • The proposed approach focuses on bridging the gap between neuroscience and Web technologies • It is currently being applied to retrieval of cardiological knowledge from the Web ADBIS'07 Varna

27. Acknowledgement This work is supported by: NSRF of the Bulgarian Ministry of Education and Science as part of the Research Project № MI - 1509/2005 ”Multimodal User and Sensor Interface in a Computer-Aided System for Cardiological Diagnosis and Intervention” ADBIS'07 Varna

28. Thank you for your attention ADBIS'07 Varna