Quality of information

1. Quality of information Considerations for library & information services in the networked world Tefko Saracevic, PhD tefkos@rutgers.edu http://comminfo.rutgers.edu/~tefko/ Tefko Saracevic

2. Table of Content • Components of quality & definitions • General problem & the Web • Information quality metrics • Information service quality metrics - usability • Methods • Conclusions Tefko Saracevic

3. Components of quality Tefko Saracevic

4. Interaction • Quality of information = content • Quality of information service = usability • Two components of quality interact • But for a user it is mostly a single experience, they blend • a service can help or hinder • assist in getting of information • or make it hard or even obstruct Tefko Saracevic

5. Quality of information What? • A pragmatic definition:The fitness for use of the information provided • key concept: fitness for use • A measure of the value which the information provides to the user of that information. • key concept: value to the user Tefko Saracevic

6. Problem • To use Web resources effectively we need both: providing or finding quality information & providing quality services • but Web information, data & databases fuzzy • structures vary widely; no consistency • constantly evolve over time • plus: Web is value neutral • indiscriminately contains both pearls & junk, truth & distortions, leading & misleading information ... Tefko Saracevic

7. Solution • Some standardized metrics to describe, recognize, & test quality • a mechanism for a more precise description & evaluation of quality things on the Web & in associated information services • Many efforts to specify metrics for information quality & for quality of information services Tefko Saracevic

8. quality Tefko Saracevic

9. Quality & metrics Quality should be: • Observable • Quantifiable, comparable, testable • Done by use of metrics:Parameters or measures of quantitative assessment used for measurement, comparison, or to track performance Tefko Saracevic

10. Metrics for quality information - how to describe, measure it General categories describing qualities of information: • Intrinsic – given internal characteristics • Context – relation to task, matter at hand • Representation – relation to user • Access – provisions provided Tefko Saracevic

11. Quality information metrics:1.Intrinsicinternal characteristics of information • Authority • Who did it? Credibility? Reputation? • Verifiability • Can be verified? • Objectivity • Trustworthy? Bias? Prejudice? Partiality? • Validity, reliability • How accurate? Believable? Correct?

12. Quality information metrics:2.Contextwithin task or matter at hand • Relevance • Relation to topic? • Appropriateness • Significant bearing on matter or task at hand? • Timeliness • Current? for the matter or task at hand • Comprehensiveness • Complete? for the matter or task at hand Tefko Saracevic

13. Quality information metrics:3.Representationwithin level & need of user • Organization • Presentation in a logical, coherent manner? • Suitability • Ease of understanding for given user(s)? • Consistency • Compatibility in presentation of different sources? • Conciseness • Compact presentation; not overwhelming? Tefko Saracevic

14. Quality information metrics:4.Accessibilityprovisions provided by system • Availability • Access? Ability to obtain information in full? • Convenience • To use? Perform actions? • Security • Restrictions? Protections? • Integrity • Adherence to ethical principles? Tefko Saracevic

15. What? Quality of information services • Linked to usability: “Extent to which a user can achieve goals with effectiveness, efficiency and satisfaction in context of use” International Standards Organization - ISO 9241-11 (1998) • Key concept: achievement of goals in context of use Tefko Saracevic

16. What? Usability “Usability is a quality attribute that assesses how easy user interfaces are to use. The word "usability" also refers to methods for improving ease-of-use during the design process.” Jacob Nielsen (usability guru) definition • Key concept: ease-of-use Tefko Saracevic

17. Essential features of usability • A user is involved • That user is doing something • That user is doing something with a product, system or other thing Tom Tullis & Bill Albert (2008). Measuring the user experience: Collecting, analyzing and presenting usability metrics. Elsevier p. 4. Tefko Saracevic

18. Essential qualities of usability • As far as users are concerned information should be • findable • understandable • utilizable • Thus information services should be built to follow these qualities • & tested to see if they indeed do that Tefko Saracevic

19. Metrics for quality information service - usability- how to describe, measure it General categories describing usability: • Effectiveness – fit for purpose • Efficiency – time for learning, work, use • Functionality – browsing, searching ... • Satisfaction – user thinking & feeling, Tefko Saracevic

20. Metrics for quality information service – usability1. Effectivenessin achieving tasks, goals • Success rate • How successful, in finding, understanding, utilizing? • Accuracy • Precision in retrieval? Amount of junk? • Completeness • Recall in retrieval? Missing? Tefko Saracevic

21. Metrics for quality information service – usability2. Efficiency in achieving tasks, goals • Learning: time, cost, effort • How much to master? • Using: time, cost, effort • How much to use? • Completion rate • Completed tasks? Tefko Saracevic

22. Metrics for quality information service – usability3. Functionality • Navigation • What provided? Hard? Easy? • Browsing, searching • What provided? Effective? • Features available • Links, relations, guides, help ...? • Outputs • Range? Tefko Saracevic

23. Metrics for quality information service – usability4. Satisfactionsubjective responses by users Satisfaction with: • Features • Interaction, process • Results • Overall experience Tefko Saracevic

24. Information Service Tefko Saracevic

25. Methods for quality studies • Libraries & digital libraries in particular are complex systems • many methods for study, evaluation appropriate • each has strengths, weaknesses • range of methods used is wide • there is no “best” method • but, no agreement or standardization on any method Tefko Saracevic

26. Methodologies used • Surveys (most prevalent) • Interviews • Observations • Focus groups • Case studies • Experiments Tefko Saracevic

27. A popular method:LibQUALAssociation of Research Libraries • Suite of services for libraries to solicit, track, understand & act upon users opinions of service quality (various fees) • A Web based survey for users • 22 core questions plus open ended comments • tools for analyzing, relating, displaying results • bundled with training & tutorials for librarians • Widely used in many countries Tefko Saracevic

28. What do digital users want from academic libraries? • Access to more digital content of all kinds & formats • Enhanced functionality: • e.g. access to full text of articles • Enhanced content: • e.g. subject information, summaries, tables of content, assistance in evaluation of resources From: Digital information seekers: How academic libraries can support the use of digital resources. Survey of 12 studies http://www.jisc.ac.uk/publications/reports/2010/digitalinformationseekers.aspx Tefko Saracevic

29. How academic libraries can meet the needs of their users? • Seamless access to a variety of resources • catalogs: more direct links • more resources from e-journals to datasets • Respond to changing user information behavior • e.g. power browsing for specific information in articles, books • Function more like search engines & popular sites – these are familiar to users Tefko Saracevic

30. How academic libraries can meet the needs of their users?... continued • Include high quality meta data – for searching, browsing • Provide guidance to students on finding & evaluating content • Advertise its brand & resources to users; demonstrate value clearly From same report http://www.jisc.ac.uk/publications/reports/2010/digitalinformationseekers.aspx Tefko Saracevic

31. A possible future • Going from machine-readable & - searchable to machine-understandable • possibly semantic Web • quality related meta data • missing in original Web architecture • So far quality is a human assessment • Future: machines assisting in determining quality Tefko Saracevic

32. Competition“everybody wants to be in information” First time in history: • Libraries are facing growing competition from many institutions & fields, & even commercial firms e.g. • many built digital libraries or provide various & innovative information services • e-books are making major inroads • What are appropriate responses? Tefko Saracevic

33. Conclusion:Value-added • For libraries & information systems, librarians & information specialists • assessing quality of information resources • & providing quality information service is the highest level of value added • Not easy to do but doable • A major problem: changing mindsets Tefko Saracevic

34. Gracias Thank you Merci Hvala Obrigado Thank you for inviting me! Grazie Tefko Saracevic

35. Presentation in Worldehttp://www.wordle.net/ Tefko Saracevic