100 likes | 327 Views
Beyond Relevance Judgments: Cognitive Shifts and Gratification. Professor Amanda Spink, Frances Alvarado-Albertorio & Jia T. Du Queensland University of Technology. Introduction. Exploring levels of evaluation by IR system users and impacts of IR interaction. Relevance level
E N D
Beyond Relevance Judgments: Cognitive Shifts and Gratification Professor Amanda Spink, Frances Alvarado-Albertorio & Jia T. Du Queensland University of Technology
Introduction • Exploring levels of evaluation by IR system users and impacts of IR interaction. • Relevance level • Cognitive shift level • Gratification level
Cognitive Shifts Level • Cognitive shift or shift in cognitive focus as triggered by the brain’s response and change due to some external force. • A shift in cognitive focus (Jacob, 2002) • Previous related interactive IR studies – moves (Fidel, 1985), shifts (Robins, 2000; Xie, 2000; Stanton, 2003). • Spink (2002) showed that all users experience some level of cognitive shifts – proposed IR evaluation measure information problem shift.
Cognitive Shifts • Spink and Dee (2007) elucidate the interactive IR as a cognitive shifting process. • All study participants reported some level of cognitive shift in their information problem, information seeking, and personal knowledge due to their Web search interaction, positive or negative shift. • Different study participants reported different levels of cognitive shift. • Some study participants’ reported cognitive shifts in various user-based variables, such as information problem or information seeking stage.
Cognitive Shifts • Interactive IR involves levels of cognitive shift in information problem, personal knowledge, or information seeking stage. • Some study participants reporting that on some levels they remained in the same stage, while shifting on other levels. • Phenomenon of cognitive shift is an important element of interactive IR. • Conducting further studies examining cognitive shifts, multitasking and cognitive coordination during interactive IR.
Gratification Level • Gratification is a basic element of human interaction with media and technologies • Uses and gratification theory used in media and technology studies (Flanagin & Metzger, 2001). • Evidence of users delaying gratification and immediate gratification during interactive processes (Mischel & Ayduk, 2004; Spink, Park & Koshman, 2006). • Spink, Park and Koshman (2006) show that Web search ordering by users a delayed gratification or immediate gratification process.
Gratification • Delayed gratification occurs when users choose to work on harder and low domain knowledge topics first. • Immediate gratification occurs when the user chooses to complete easier or high domain knowledge topics first. • Need for further research exploring the role of gratification during interactive information retrieval.
Further Research • Explore the nature and role of various processes during interactive IR that can form the basis of evaluation measures, including: - Cognitive shifts and impacts during interactive IR - Gratification impacts during interactive IR • Need to further explore the cognitive impacts of interactive IR beyond relevance judgments.
References • Du, J. T. (2009). Multitasking, Cognitive Coordination and Cognitive Shifts During Web Searching. Unpublished dissertation, Queensland University of Technology. • Spink, A. (2002). A user centered approach to the evaluation of Web search engines: An exploratory study. Information Processing and Management, 38(3), 401-426. • Spink, A., & Dee, C. (2007). Cognitive shifts related to interactive information retrieval: A replication study. Online Information Review, 31(6).
QUESTIONS? Thank You