Evaluating Library Automation Software:

1. Evaluating Library Automation Software: A View from the Classroom Shelly Warwick Queens College, CUNY National Online Meeting 2001 Shelly_Warwick@qc.edu ©2001 - Shelly Warwick

2. Focus of This Presentation • Benefits and challenges of teaching evaluation methodologies in the class • How student problems in mastering selection methodology are relevant to practitioners

3. A Change in Learning Environments • Early selectors of automation systems had to learn on the job and invent the rules and procedures • Later selectors depended on advice of early selectors offered in the literature or in workshops • Current library school students can be introduced to selection methods and criteria in the classroom

4. Context of Observations • Queens College Graduate School of Library and Information Studies • Course on selecting appropriate media and technology • Large unit on selection of automation systems • Required course for those in school library media track • Many students currently working in elementary school libraries (M.L.S. currently not required in New York State) - some charged with selection an automation package • Other students employed or trainees in public libraries or academic libraries • Some with no library experience • 95% of students have only used public functions of automation system before taking the course

5. Approaches Utilized • Discussion of history, theory and functions of library automation systems • Presentation of a methodology for evaluating and purchasing automation technology • Hands-on exploration and evaluation of automation packages • Vendor demonstrations

6. Student Approaches to Evaluation Projects • Minimum Effort • Very little time spent with software • Reliance on vendor statements or reviews • Seeking Simplicity • Overwhelmed by large manuals and complex programs • Pick systems to evaluated based on smallest amount of instructional materials • Focus on three or four functions - ignore many areas required to be addressed by assignment

7. Student Approaches to Evaluation Projects - II • Involved • Visits vendor sites • Reads reviews • Fully explores programs • Notice what is missing as well as what is present • Often those with the least experience question assumptions and business as usual approaches that are not well thought out • Some areas required in assignment still ignored

8. Preferred Method of Learning • Vendor Demos! • In Class (scheduled after evaluation project due) • Exhibits • Students do not question vendor - despite instructions to challenge assumptions and request demonstration of functions that are not part of canned presentation

9. Exceptional Reliance on Demo • Preference of demonstrated system in selection papers • Purchase of demonstrated system • Failure to visit installed site

10. Implications For Libraries • Novice students assigned to select based on the course were able to make a selection that satisfied their administrators • Selection skills can be taught • Despite being informed that canned demonstrations are not to be trusted students were unwilling to ask questions that might make them seem ignorant or rude • Selection teams should meet prior to demos and assign types of questions to individuals • Easier to use programs with less functionality preferred over more difficult programs with more features • Usability prime consideration in system selection

11. Implications For Libraries -Continued • Students focused on functions and tasks which were relevant in their current position • The more experienced the selector the more demanding the criteria • Individuals with different areas of expertise and types of experience should participate in the selection process • New selectors questioned assumptions • Someone new to the profession should be in selection group • Unwillingness to explore complex systems • Systems are often adopted that are easy to use but do not meet more advanced needs

12. Implications for Vendors • Students viewed vendors as “authority” not a salesperson - worked with one they liked the most • Vendor personality and presentation skills important • Students evaluated and/or selected packages from vendors that were easy to contact and readily provided information • Poor pre-sale communication is viewed as an indication of poor after sale support - a good website is a must • Demos were the key to selection • Providing access to full version via demo disks or the web attracts customers

13. Implication for Vendors - Continued • Students/graduates charged with selecting a system generally selected one they had evaluated in class • Working with library schools and providing free full versions of automation software or demos with access to administrative functions is a good investment

14. General Recommendation • A professional group of SIG involved in automation should develop test data sets that reflect the volume and structure of data for various size and types of libraries • Such data could be used to more effectively compare library automation systems

15. Conclusion • Teaching the evaluation of automation software in library school benefits all • Students • understand the selection process • familiar with evaluation criteria • Vendors • contact with potential customers • Libraries • new employees with an understanding of what automation systems can do and capable of participating in next round of selection

16. Discussion - Questions

17. Thank you!!! Shelly Warwick Shelly_Warwick@qc.edu