Yan Zhang University of North Carolina, Chapel Hill Email: yanz@email.unc

1. Measuring mental models: Rationale and instrument Yan Zhang University of North Carolina, Chapel Hill Email: yanz@email.unc.edu Peiling Wang The University of Tennessee, Knoxville Email: peilingw@utk.edu The concept of mental models Method—Survey instrument Introduction Mental modelrefers to users’ conceptual representation of the system. The Web as an IR system is complex and abstract. End users must have a mental model of the system when interacting with it. Mental models are incomplete, limited, unscientific, unstable and deceptively simple, but they are vitally important. It is the mental models that enable users to interact with and learn in trial-and-error about systems. The purpose of this study is to develop and test instruments for measuring users’ mental models of the Web as an IR system. Scales: 5 components: • Experience • Web space • Web structure • Search functions • Interface Reliability • A subset of reversed questions • Internal consistency: homogeneity of indicators in each scale Validity • Criteria related validation: test two groups with apparent different level of mental models in terms of sophistication • Content validity: expert review Research goals • User view: • appearance • documentation • structure • operations Interface • Help build systems that are able to provide meaningful and context-sensitive cues to facilitate users build mental models • Inform design of congruent interfaces that can anticipate users’ next movement to provide adequate support • Develop learning tools and curriculums to help users move from novice to advanced levels In the ideal world, the three models match with each other. Mental model Work in progress Conceptual model Framework for measuring MMs of the Web • By observing users interaction with the Web to explore how users apply mental models in interaction with systems. • Mental models are unstable, changing with the increasing of users’ knowledge states. Future research could be conducted to explore how mental models change over time and what are the characteristics of the changes. • More research is needed to probe into the format of mental models and how users drive mental models. Search engines Postulations Web Space: Information Objects Collections Searching:Process input Generate output • Mental models are based on prior experiences with similar systems using analogy or metaphor. • There are gaps between mental models and a system’s conceptual model. A system has an underlying conceptual model that is often hidden from its users. • There are gaps between the system’s conceptual model and its user views or external models. There are often more than one user views targeting different users. Typically, one for novices and one for advanced users. Interface Navigation tools Access methods Access results Structure:Hyperlinks Indexing Representing mechanism Web Space Search engines References Interface User Allan, J. (Editor). (2003). Challenges in Information Retrieval and Language Modeling: Report of a Workshop held at the Center for Intelligent Information Retrieval, University of Massachusetts Amherst, September 2002. SIGIR Forum, 37(1), 31-47. Norman, D. A. (1983). Some Observations on Mental Models. D. R. Gentner, & A. L. Stevens Mental Models (pp. 7-14). Hillsdale, NJ: Erlbaum. Wang, P., Hawk, W. B., & Tenopir, C. (2000). Users' interaction with World Wide Web resources: an exploratory study using a holistic approach. Information Processing and Management, 36, 229-251