Quality Assessment Program as a Contribution to the SIBi/USP Management

2. Quality Assessment Program as a Contribution to the SIBi/USP Management Cybelle de Assumpção Fontes caf@usp.br IATUL – 2006Porto, Portugal University of São Paulo Integrated Library SystemSão Paulo, SP - Brazil

3. Quality Assessment Program as a Contribution to the SIBi/USP Management Maria Imaculada Cardoso Sampaio (Coordinator) Adherbal Caminada Netto Adriana A. Barreiros Adriana Cybele Ferrari Cybelle Assumpção Fontes Márcia Elisa Garcia de Grandi Maria Cristina Olaio Villela Rosa Maria F. Zani IATUL – 2006Porto, Portugal

4. Who we are:

5. University of São Paulo • A public, autonomous institution, funded by the State of São Paulo, created in 1934. • The largest institution of higher education and research in Brazil, and the third in size in Latin America. • Teaching units in São Paulo, Bauru, Piracicaba, Pirassununga, Ribeirão Preto and São Carlos. • Numbers • 202 Undergraduate Courses • 544 Graduate Courses • 77.205 Students Enrolled • 5.078 Faculty Members • 15.008 Staff • (Database 2004)

6. University of São PauloIntegrated Library System (SIBi) • 40 Libraries • Collection • Books 1.827.872 • Theses and Dissertations 196.679 • Journals (Current Titles) 20.985 • Users 126.192 • SIBiNet – SIBi/USP Services Network • DEDALUS - Bibliographic Database (OPAC) • Access to electronic journals • Access to data bases • Rare Books online • Digital Library of Dissertations and Theses • (Database 2004)

7. BackgroundUsers and Libraries • Library focus • Perceptions and needs of users • Information systems can better meet their users’ needs with services more suitable and useful to their demands • Users’ opinion as a tool in the information system management

8. BackgroundQuality and Libraries • SERVQUAL • Wide application for search service quality • Gap between user’s expectation and perception(what is really provided) • Dimensions for identify quality • Tangibles: appearance of physical facilities, equipment, personnel, and communication materials • Reliability: ability to perform the promised service dependably and accurately • Responsiveness: willingness to help customers and provide prompt service • Assurance: knowledge and courtesy of employees and their ability to convey trust and confidence • Empathy: the caring, individualized attention provided to the customers

9. BackgroundQuality and Libraries • LibQUAL+ • SERVQUAL application to the library environment • Association of Research Libraries (ARL) and Texas A&M University Libraries (EUA) • Users’ perceptions of excellence, current and minimal level of service acceptable • Longitudinal data and benchmark

10. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • PAQ • Structured in 2002 • Focused on the users’ perception • Based on SERVQUAL/LibQUAL+ dimensions: • tangibles • reliability • responsiveness • assurance • empathy • Search • First Phase: 2002/2004 • Second Phase: 2005/2006

11. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • First Phase 2002/2004 Users’ opinion regarding products and services provided by the libraries of the System • Five electronic questionnaires • Questions based on SERVQUAL’s dimensions • Personal interview • Open questions: what the user likes least, likes most and what he/she would like to change in the library

12. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • First Phase 2002/2004 Results • Local actions reverted on behalf of the users • Subsidized the SIBi/USP strategic planning from 2003 • Proposal of a new Program’s phase

13. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 Users’ needs – SIBiNet SIBi Service Network Resources • Interview based on the Critical Incident Technique • Electronic questionnaire

14. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 • Interview based on the Critical Incident Technique • Procedure for collection of direct observations of human behavior (Flanagan, 1954) • Incidents observed which present special significance and to find systematically defined criteria • A valid critical incident that can define users’ needs must consider two characteristics: • be specific and refer to a clearly demarcated scene • be able to describe either the service provider in behavioral terms, or the product and services, with particular adjectives

15. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 • Interview based on the Critical Incident Technique • Users in libraries in which PAQ participants work • Analysis and selection of main critical incidents • Critical incidents and SERVQUAL dimensions • tangibles – broken links • reliability – lack of full theses • responsiveness – training on SIBiNet • assurance – site updating • empathy – suitable layout

16. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 • Electronic questionnaire (at SIBiNet) • Based on the categorized information (Critical Incident) • Structure • Access to the site – compatibility and network • Contents – database, periodicals • Services – another portals • DEDALUS • Users’ categories: USP community, non-USP community and SIBi/USP technical team • Open form: free expression

17. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 Results • 2304 questionnaires • Open form – answers analyzed • DEDALUS – weak points • Site – weak points • Questionnaire • Level of dissatisfaction – constant in the three users’ groups • Global analysis of dissatisfaction – Pareto’s Chart

18. + - Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Second Phase 2005/2006 Pareto’s Chart (Disagreement + Total disagreement)

19. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Final Considerations • First phase Enhancements regarding products and services were implemented • Supplementation of the allocated budget for the acquisition of bibliographic material; • renewal and updating of the libraries’ technological facilities; • extension of service hours in some libraries; • remodeling of physical spaces in the library; • changes in the libraries access control systems; • grounds for the Project of replacement of the Libraries’ database management software (DEDALUS); • inclusion of new information sources in several knowledge areas; • definition of a continuous capacity building of the teams, focused on quality.

20. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Final Considerations • Second Phase Supported proposal for the overall change of the SIBi/USP site.

21. Quality Assessment Program (PAQ)Integrated Library System (SIBi) • Final Considerations • The proposed continuous assessment program was proven feasible in the management and has been adopted as a managerial tool. • The credibility and respect that the information system has before its community can only be certified by the actual system’s auditor: the user.

22. Thank you! dtsibi@org.usp.br

23. Good luck!