Cognitive & Context Modeling for Interactive IR

1. Cognitive & Context Modeling for Interactive IR Nicholas J. BelkinRutgers University belkin@rutgers.edu

2. What’s Cognitive Modeling in IIR? • “System’s” model of the user’s “information need” (user model) • User’s model of the system (system model) • Presentation to the user of the system (conceptual model) • and so on

3. What’s Context Modeling in IIR? • “System’s” understanding of what led the searcher to engage in the IIRS • System’s understanding of characteristics of the individual searcher • System’s understanding of the searcher’s “environment”

4. IIR and User Modeling • If we construe user modeling in the IR domain to be equivalent to representation of the user’s information “need”, then IR has, from its inception, been an inherently user modeling activity • But IR as a discipline has paid relatively little attention to how “best” to construct this kind of user model, at least until quite recently

5. IIR and User Modeling • But IR, through the essential concept of relevance, has always understood that representation of the information “need” is only a part of the user model • For a variety of practical, technical and experimental reasons, the other aspects of the user model have been relatively little explored

6. A Brief History of User Modeling in IR • When IR was a strictly manual operation, human mediators (e.g. reference librarians) often constructed complex models of the user’s information problem, goals and situation, and used these models as the basis for offering information or advice • see, e.g. Taylor, (1968)

7. Taylor (1968) • Four levels of information need (question formulation) • visceral; inexpressible • conscious; partially expressible • formalized; expressed • compromised; expressed in system terms

8. Taylor (1968) • Five “filters” negotiated by intermediary and user • Determination of subject • User’s objective and motivation • Personal characteristics of user (including previous searching activity) • Relationship of need description to file • Anticipated or acceptable answers

9. More History • When IR began to become mechanized (late 1950s, early 1960s) emphasis shifted to elaborate query specification as the major “user modeling” activity • Primarily a function of the comparison techniques and the complexity of the query specification languages • Implemented in SDI systems

10. SDI Systems (still history) • SDI worked by having users construct long- term information need representations, which are periodically compared to new information objects • SDI “profiles” were originally only modified explicitly by users, in concert with information professionals, on the basis of the users’ evaluations of the search results

11. Ad hoc IR • Ad hoc IR was assumed to be dealing with the problem of helping a user to find information related to a current, specific problem • Ad hoc IR operated within a single information-seeking episode, often construed as a single query-response cycle • Ad hoc IR attempted to represent current, rather than long-term information needs

12. Static Modeling of Need • Characterizes the bulk of need representation research in IR • Attempts to enhance the expression of need (the query, i.e. Taylor’s compromised need) • Examples include • Pseudo-relevance feedback • Thesaurus-based query expansion • Corpus-based query expansion

13. Dynamic Modeling of Need • Two general solutions to dynamic modeling of the ad hoc information need • interactive IR in which the user reformulates the query according to system results • relevance feedback (RF), in which the search system reformulates the query according to responses to system output elicited from the user

14. Dynamic Query Reformulation • Searcher does this unaided, except through interpretation of search results • Searcher is encouraged to expand query (cf. Kelly, Dollu, Fu, 2005) • Searcher is offered possible means for reformulation (e.g. Koenemann & Belkin, 1996; papers in the TREC HARD track)

15. RF (A Quick Reminder) • Relevance feedback was first proposed by Rocchio in 1966 (Rocchio, 1971) • Relevance feedback attempts to generate the “ideal” query, which can be thought of as the “perfect” model of the information need

16. “Queryless IR” • Proposed by Oddy (1977) as means to: • avoid specification of information need • account for evolution and change in user’s understanding of information problem and relevant documents • Search system constructs model of user’s interest in terms of subgraphs of the information resource, through interpretation of user’s interaction with information objects • Oddy quite explicit about this not being a user model

17. The “Cognitive” Turn in IR • Early to mid 1970s, IR began to consider modeling not of “needs”, but rather of “knowledge states” • B.C. Brookes’ “fundamental equation of information science” (Brookes, 1980) • Belkin’s Anomalous States of Knowledge (Belkin, 1980) • Although there was recognition of other factors, predicting and understanding changes in states of knowledge was key

18. The “Problematic Situation”, etc. • Wersig (1979) proposed that ASK was only one aspect of why people engage in IR systems • Goals, intentions, uncertainties, context were also key • Schutz & Luckmann’s (1973) problematic situation an overarching view • Dervin’s (1992) sense-making explicitly incorporates aspects other than knowledge

19. Modeling the Intermediary • 1980s, attempts to “automate” the human search intermediary (Vickery & Brooks, 1987) • Functional specification of the intermediary • MONSTRAT (Belkin, Seeger & Wersig, 1983), I3R (Croft & Thompson, 1986), CODER (Fox, 1987) • More generally, DEBIS (Belkin, et al., 1987)

20. Modeling beyond “Need” • Modeling IR experience, domain knowledge, etc. (e.g. Brajnik, Guida, Tasso, 1987) • To support more effective interaction, to tailor search system response to (presumed) user goals • Accomplished primarily through direct elicitation

21. Distributed Expert-Based Information Systems • Models of user’s: • Topic of interest • Goals • Previous activities • Preferred dialogue mode • Knowledge of topic, and of system • Preferred response types …. (reminiscent of someone?) • Supported by research on relevance criteria

22. Too Much Modeling? • Spärck Jones (1989) points out difficulties associated with detailed user modeling • I3R, CODER both suffered from computational and communication complexity

23. Or Too Much All at Once? • DEBIS-style modeling tried to construct models early in the interaction • Much work based on user-intermediary interaction examples of 1970s-80s • Shouldn’t models be constructed incrementally? • Interaction for modeling interferes with interaction for searching (e.g. Cool, 1998)

24. The Interactive Turn • Support direct interaction with information objects • Refrain from modeling at all (Landauer; others) • Information retrieval as interaction with texts • Return to THOMAS (Oddy, 1977)? Support for “berrypicking” (Bates, 1989)?

25. Model Building through Interaction with Texts • Nordlie’s (1999) observations of public service encounters in libraries • Model building by observing and interpreting interactions with texts • Minimal initial model, little explicit model-building interaction • Model building through implicit sources of evidence (behaviors)

26. What has been Modeled from Observation of Behavior? • Information “need” • Implicit relevance feedback • Previous searching behaviors • Previous use behaviors • cf. Kelly (2005) • Can (should) models be created based on relevance criteria other than topicality? Or on other factors?

27. Some Categories of Relevance Criteria* • Topic • Type of content • Authority • Format • Presentation • Relationship to self and goals * From Belkin, Cool, Frieder & Kantor (1993)

28. Personalized (Contextualized) IR* • TREC HARD track • Searcher’s knowledge, desired document types (Allan, 2005) • Tasks and goals • Context around the searcher • Time, space • Social context * cf. Belkin (2006)

29. IIR and Cognitive Modeling (no more history) • Cognitive modeling of “need”, “intention” still most common • Cognitive modeling of individual characteristics becoming significant • Here, affect considered as part of cognitive model

30. IIR and Contextual Modeling • Contextual modeling of goals, tasks recognized as important • Contextual modeling of environmental factors being used • Contextual modeling of social factors becoming recognized as significant

31. Current Modeling Approaches in IIR • Implicit rather than explicit • Through the course of the interaction • Considering search episode as a whole • Integration of “ad hoc” and “filtering” approaches, i.e. long term and short term, all dynamic

32. References • See the References file in this section.