What is a Successful Digital Library?

1. What is a Successful Digital Library? ECDL 2006, Alicante, September 18, 2006 Rao Shen, Naga Srinivas Vemuri, Weiguo Fan, and Edward A. Fox fox@vt.edu http://fox.cs.vt.edu

2. Acknowledgements (Selected) • Sponsors: NSF grant ITR-0325579, ASOR, CWRU, ETANA, Vanderbilt U., Virginia Tech • Faculty/Staff: Lillian Cassel, Debra Dudley, Manuel Perez, … • VT (Former) Students: Aaron Krowne, Ming Luo, Fernando Das Neves, Ricardo Torres, Hussein Suleman, …

3. Acknowledgements (Selected) • Karen Borstad,MPP • Giorgio Buccellati, UCLA • Douglas Clark, Walla Walla College • Joanne Eustis, CWRU • Nick Fischio, CWRU • Israel Finkelstein, Tel-Aviv University • Paul Gherman, Vanderbilt U. • Andrew Graham, U. Toronto • Tim Harrison, U. Toronto • Larry Herr, Canadian University College • Christopher Holland,LRP • Paul Jacobs, Mississippi State U. • Douglas Knight, Vanderbilt U. • Stan LaBianca, Andrews U. • David McCreery, Willamette U. • Eric Meyers,Duke U. • Adam Porter, Illinois College • Jack Sasson, Vanderbilt U. • Tom Schaub,Indiana U. of Penn. • Randall Younker, Andrews U.

5. ETANA-DL Website

6. ETANA-DL’s Member Collections

7. Outline • Prior work • DL success model • From end user perspective • Case study • Conclusion

8. Prior Work on Measuring DL Success inspection of NCSTRL Usability of DLs has an example evaluation of ACM, IEEE-CS, NCSTRL, and NDLTD evaluation of ADL evaluation of ADEPT Intention to re/use predict Technology acceptance model has an example Venkatesh system usage DeLone et al. has an indicator IS success model has an example Seddon Ellis Information seeking behavior model has an example Kuhlthau DL quality model has an example Gonçalves

9. intention to use system performance expectancy system usage effort expectancy social influence facilitating conditions Behavioral Attitude & Intension to Use— Venkatesh Model of IT Adoption Unified Theory of Acceptance and Use of Technology

10. Venkatesh Model of IT Adoption • Performance expectancy: • perceived usefulness, extrinsic motivation, job-fit, relative advantage, and outcome expectations • Effort expectancy • the degree of ease associated with the use of system • Social influence • Subjective norms, social factors, and image • Facilitating conditions • the degree to which an individual believes that an organizational and technical infrastructure exist to support the system

11. DeLone and McLean Model of IS Success System Quality Use Individual Impact Organization Impact Information Quality User Satisfaction

12. Seddon Model of IS Success Net Benefits to: Individuals Organizations Society System Quality Perceived Usefulness Information Quality User Satisfactions

13. Outline • Prior work • DL success model • From end user perspective • Case study • Conclusion

14. DL Success Model • 5S and minimal DL • Synthesize • IS success and adoption models (see above) • Information life cycle model (Borgman et al.) • 5S-based DL quality model (Gonçalves et al.) • Information-seeking behavior models (Ellis’ and Kuhlthau’s) • From end user perspective

15. Informal 5S & DL DefinitionsDLs are complex systems that • help satisfy info needs of users (societies) • provide info services (scenarios) • organize info in usable ways (structures) • present info in usable ways (spaces) • communicate info with users (streams)

16. 5S and DL formal definitions and compositions (April 2004 TOIS)

17. Information Life Cycle Borgman et al.: Workshop Report on Social Aspects of Digital Libraries: http://www-lis.gseis. ucla.edu/DL/

18. Information Life Cycle Creation Active Authoring Modifying Social Context Using Creating Organizing Indexing Retention / Mining Accessing Filtering Storing Retrieving Semi- Active Utilization Distributing Networking Inactive Searching

19. searching, browsing, recommending accessing, filtering completeness, conformance accuracy, Active accessibility, preservability Semi-active preservability, authoring, modifying, describing organizing, indexing storing, archiving, networking similarity, timeliness, creation distribution utilization seeking pertinence, significance, relevance Gonçalves et al. accessibility, Inactive timeliness

21. E1:starting K1: initiation K6: presentation E2: chaining E3: browsing E4: differentiating E5: monitoring E6: extracting K2: selection K5: collection K3: exploration K4: formulation E: Ellis’ model K: Kuhlthau’s model completeness, conformance accuracy, Active accessibility, preservability Semi-active preservability, authoring, modifying, describing organizing, indexing storing, archiving, networking similarity, timeliness, creation distribution utilization seeking Inactive DL Success Constructs

22. relevance adequacy timeliness reliability understandability scope ease of use accessibility user interface joy of use reliability DL Success Model information quality (IQ) information quality (IQ) performance expectancy (PE) behavioral Intention to (re)use satisfaction system quality (SQ) social influence (SI)

23. collection metadata catalog digital object metadata specification repository DL Concepts Regarding Information associated with consist of consist of associated with consist of

24. Digital Library IQ digital object accessibility understandability preservability relevance pertinence timeliness relevance reliability similarity adequacy significance timeliness metadata specification accuracy completeness conformance collection completeness impact factor completeness catalog consistency completeness repository consistency Information Quality (IQ) digital object metadata specification catalog collection adequacy repository scope

25. relevance adequacy timeliness reliability understandability scope ease of use accessibility user interface joy of use reliability DL Success Model information quality (IQ) performance expectancy (PE) behavioral Intention to (re)use satisfaction system quality (SQ) system quality (SQ) social influence (SI)

26. service accessibility composability reliability reusability ease of use extensibility joy of use efficiency effectiveness reliability screen design navigation Digital Library SQ System Quality (SQ) performance expectancy user interface

27. Outline • Prior work • DL success model • From end user perspective • Case study • Conclusion

28. Case Study • Part of requirements analysis forETANA-DL • Email interviews with 5 prestigious archaeologists • Face to face workplace interviews with 11 archaeologists • Associate the 4 constructs of DL success model with the activities occurring in the seeking and utilization phases

29. DL Success Constructs Associated with Seeking and Utilization Phases

30. DL Success Constructs Associated with Seeking Phase • E1: Starting’ activity in Ellis’ model (K1: ‘initiation’ stage in Kuhlthau’s model) • Social Influence (SI) — DL visibility • Publicize existence of a DL • Provide a DL alert service

31. DL Success ConstructsAssociated with Seeking Phase • E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) • Information Quality (IQ) • System Quality (SQ) • Performance Expectancy (PE)

32. DL Success ConstructsAssociated with Seeking Phase • E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) • Information Quality (IQ) • Adequacy (degree of sufficiency and completeness) of DL collections and metadata catalogs • Scope of DL repository

33. DL Success Constructs Associated with Seeking Phases • E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) • System Quality (SQ) • Ease of use • Joy of use

34. DL Success Constructs Associated with Seeking Phases • E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) • Performance Expectancy (PE) • Usefulness

35. DL Success ConstructsAssociated with Seeking Phases • E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) • System Quality & Performance Expectancy • DL interface: screen design & navigation

36. DL Success Constructs Associated with Utilization Phase • K4-K6: ‘formulation’, ‘collection’, and ‘presentation’ stage in Kuhlthau’s model • Information Quality • information accuracy • information accessibility

37. Outline • Prior work • DL success model • From end user perspective • Case study • Conclusion

38. Conclusion • Lay the foundation for defining success of DLs from the view of DL end users • Assume a multi-theoretical perspective • Synthesize many related research areas in terms of theory and empirical work • Explicate and illustrate our approach by a case study with ETANA and usability • Connect with other work on DL quality: led by Emory funded by IMLS, DELOS …

39. Questions?Comments? See http://fox.cs.vt.edu/talks/2006/ 20060918ECDLsuccess.ppt Thank You!