Evaluating Electronic Resources

1. Evaluating Electronic Resources Charleston Conference 2005

2. Evaluating Electronic Resources:A Bird’s Eye View • Welcome & Introductions • Audrey Powers • Evaluating Electronic Resources: A Cooperative Process with Objective Results • Break • George Machovec • Consortial purchasing • Gold Rush (ERM and content analysis) • The Charleston Advisor & Snapshot Reviews • Q & A

3. Evaluating Electronic ResourcesA Collaborative Process with Objective Results • Evaluate electronic resources for acquisition, retention and withdrawal purposes • Identify essential criteria • Customize the evaluation form • Administer the evaluation process manually or electronically

4. If you use this process or these slides, please ask for permission and give me credit. Thank you. • Audrey PowersResearch and CollectionsUniversity of South Florida4202 E. Fowler Avenue, LIB122Tampa, FL 33620813-974-9001(Phone)813-974-9875 (Fax)apowers@lib.usf.edu

5. This evaluation process was developed to • Guide evaluators • Collaborative process • Objective results

6. Enables institutional evaluations • Acquisitions • Retention • Withdrawal • Comparisons

7. Advantages • Institutional objectives are met • Adaptable for a variety of user populations • Electronically administered • Customizable

8. Evaluate Electronic Resources • Databases • eJournals • eBooks • Open Access resources • Internet sites • Hardware • Software • Courseware

9. Evaluators • Opportunity to query a diversity of users • Students • Faculty • Administrators • Librarians • Staff

10. Institutions • Variety of institutions • Libraries • School • Public • Academic • Special • Schools • Businesses • The Charleston Advisor

11. Considerations • Purpose • Type and size of institution • Extent of existing resources • Audience • Observed use patterns • Desired results • Project timeline • Budget allocations

12. Used for • Databases • Entire collection/Discipline specific databases • Trial databases • Single trial/multiple trials • Database comparisons • Trial database/existing database

13. Why do it? • Cut in funding • Rising costs • Duplication • Librarians have “favorites” • Some databases not used

14. Why do it? • 110 databases • 80 databases are free or reduced cost due to consortial arrangements • $65,400.00 • 20% of the operating budget University of Montevallo

15. Why do it? • 512 databases • 200 databases are free or reduced cost due to consortial arrangements • $852,680.00 • 17% of the operating budget University of South Florida

16. Why do it? McCracken, Peter. "A comparison of print and electronic journal holdings in academic and public libraries." Libri 53.4 (2003): 237-241.

17. Expenditures • Journals $1,528,580.00 • Databases 860,556.00 • Books 152,920.00 • CD-ROMs 2,206.00 • Total $2,544,262.00 University of South Florida

18. Three Step Process Prepare Gather Data Results

19. Phase I PREPARE

20. Identify Known Data • Cost (actual cost per year) • Use (annual use statistics) • Cost per use • Duplication (duplication of journal titles) • Peer comparisons (availability at peer institutions)

21. Develop Evaluations • Content • Unique content • Ease of use • Instruction • Overall quality • Need • Comments

22. ASK QUESTIONS ?

23. Content • Is the content and reading level appropriate for the intended audience? • Is the coverage selected or comprehensive? • Is the content substantial? • Is this the only resource that covers this topic?

24. Unique Content • Are the journal titles unique to the database? • Is the search interface common or unique? • What are its special features?

25. Ease of use • Is the format and layout easy to navigate? • Is a standard search methodology in use? • What retrieval methods are available? • Is access to the database and search screen easy?

26. Instruction • Is the database easy to teach? • Is the number of simultaneous users limiting? • Does the licensing agreement restrict class instruction? • Is it useful for assignments?

27. Overall quality • Are there many errors? • Is the database often inaccessible? • Is the documentation useful, well written and easy to navigate? • Are use statistics available?

28. Need • What is the value to the institution? • Is it required? • Does it fulfill a need?

29. Comments • Gives the evaluators an opportunity to express opinions

30. DATABASE EVALUATION FORM Database ___________________________________________________________ URL ___________________________________________________________ Evaluator___________________________________________________________ Known Data Cost: Use: Cost/use: Duplication: Peer comparisons: Evaluations Content: 0 1 2 3 4 Unique Content: 0 1 2 3 4 Ease of use: 0 1 2 3 4 Instruction: 0 1 2 3 4 Overall quality: 0 1 2 3 4 0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory Need: ____Inappropriate _____Useful ____Essential Comments:

31. FACULTY EVALUATION FORMPlease complete and submit this form. Access the electronic resource by clicking on the hyperlink. Electronic Resource Known Data Cost: Use: Cost/use: Peer comparisons: Evaluations Content: 0 1 2 3 4 Unique Content 0 1 2 3 4 Ease of use: 0 1 2 3 4 Instruction: 0 1 2 3 4 Overall quality: 0 1 2 3 4 0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory Need: ____Inappropriate _____Useful ____Essential Comments:

32. STUDENT EVALUATION FORMPlease complete and submit this form. Access the electronic resource by clicking on the hyperlink. Electronic Resource Content: 0 1 2 3 4 Unique Content: 0 1 2 3 4 Ease of use: 0 1 2 3 4 Assignment use: 0 1 2 3 4 Overall quality: 0 1 2 3 4 0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory Need: ____Inappropriate _____Useful ____Essential Comments:

34. Phase II GATHER DATA

35. Gather essential data • Obtain Known Data • Distribute evaluation form(s) • ConductEvaluations • Compile data

36. Phase III RESULTS

37. Report Results • Collate results • Present results • Recommend an action plan

38. In Summary • Prepare • Determine criteria • Develop evaluation form • Create web forms e • Create a database e • Distribute/publish • Gather Data • Distribute evaluation forms • Email evaluation forms/Publish on web page e • Conduct evaluations • Evaluations are completed online e • Compile results • Compile the results in Access > Export to Excel e • Results • Collate the results • Manipulate the data in Excel e • Determine the average for each criterion e • Determine the composite score e • Present the results • Recommend an action plan

39. eBenefits • Customization • Distribution • Participation • Response rate • Collation of data

40. The eProcess

41. The eProcess: An Overview • Create web forms • Create a database • Compile results in Access • Export results to Excel • Manipulate data in Excel • Determine average for each criterion • Determine composite score

42. HOW TO DO IT

43. Create Web Forms • Create a new page

44. Create web forms Place cursor in the body of the page Go to Insert > Form > Form Hit Enter 2x

45. Create Web Forms Go to Insert > Form > Option Button, etc.

46. Create web forms Click 2x on Option Button and complete Group name: and Value: fields

47. Create web forms

48. Create web forms • Create a confirmation page

49. Create web forms Preview confirmation page in Browser Copy URL of confirmation page

50. Create database Right click in the body of the form Select Form Properties