1 / 32

Approaches and techniques for measuring impact

What difference does your library make to teaching and learning?. Approaches and techniques for measuring impact. Philip Payne, Librarian, Birkbeck University of London. What makes a good library?. Measures of service quality. From measures of service quality….

ksorenson
Download Presentation

Approaches and techniques for measuring impact

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What difference does your library make to teaching and learning? Approaches and techniques for measuring impact Philip Payne, Librarian, Birkbeck University of London

  2. What makes a good library?

  3. Measures of service quality

  4. From measures of service quality… …to assessment of impact

  5. What makes a good library? How much good does a library do?

  6. How much good does a library do?

  7. Possible measures of impact

  8. Are we making a difference? Can we demonstrate it?

  9. Why the need to demonstrate impact? • Libraries are expensive. Do they provide value for money? • Libraries are changing. Are we making the right choices?

  10. How does impact differ from outcomes? • Outcomes: the effects of the library of its services on individual users or users collectively • Impact: how these outcomes link with the mission and strategic goals of the host institution

  11. Some thoughts when assessing impact • Not all impacts are positive • Not all impacts are intended • Not all impacts are immediate • Difficulty in separating library impact from other influences

  12. “Assessing impact is not easy and it is not an exact science. We are dealing with a changing environment where people, services, and needs are constantly evolving. Any research will inevitably provide a snapshot of what is happening at a particular point in time.” Everest and Payne, The Impact of libraries on learning, teaching, and research: report of the LIRG seminar held in Leeds, 30th. October 2001. Library and Information ResearchNews, 25 (81), Winter 2001, p.21

  13. Some further thoughts when assessing impact • What are you trying to achieve? • Who are you trying to influence? • Who are you going to involve? • What resources are you going to need - time/expertise/costs? • How are you going to use the findings?

  14. Who was involved in the LIRG/SCONUL Impact Initiative?

  15. Phase 1 Institutions Birkbeck University of London Glasgow Caledonian University Who was involved in the LIRG/SCONUL Impact Initiative? University of Leeds University of Gloucestershire University of Teesside University College, Chester Open University Leeds Metropolitan University Northumbria University

  16. Phase 2 Institutions University of Stirling University of Birmingham Southampton University Who was involved in the LIRG/SCONUL Impact Initiative? Bournemouth University University College Northampton Brunel University University of Edinburgh Newman College University of Glasgow King’s College London University of the West of England

  17. Impact of an online induction tutorial [Birkbeck] E-resources: impact on students’ skills development and collaboration with academic staff [Chester] Impact of an online information skills tutorial [Gloucestershire] Information and IT skills: impact on confidence and competence in independent and lifelong learning [Northumbria] Information literacy strategy: awareness of the strategy and its impact [Glasgow Caledonian] Impact of library services to partner institutions [Teesside] Information literacy: impact on students’ skills development and collaboration with academic staff [Leeds] Impact of library support on the research process [Warwick] Information literacy strategy: awareness of the strategy and its impact [Leeds Metropolitan] Information literacy programme: impact on information literacy amongst students [Open University]

  18. Library support for non-traditional students of modern languages University of Southampton Equality of access to e-resources Bournemouth University Increasing amounts spent on electronic information services (EIS) each year University of the West of England Information skills for research postgraduates Brunel University Institutional Repository & Scholarly Communications Advocacy University of Birmingham Subject-specific electronic resources & the research process University of Glasgow Integrated enquiry desks King’s College, London Support for widening participation University College Northampton Subject-specific electronic resources & the research process University of Edinburgh Single point of access for IT and Library enquiries Staffordshire University Support for widening participation Newman College Student use of subject resources web pages University of Stirling

  19. Stages in assessing impact • Choose an intervention where library seeks to assess impact • Specify objectives for library intervention • Develop success criteria against which a judgement can be made as to whether the objectives have been met • Identify evidence that needs to be collected in respect of these success criteria • Select appropriate data collection methods to collect the evidence • Collect and analyse the data • Present the results

  20. Stages in assessing impact

  21. An exampleInformation literacy framework Objective To equip students with the skills to become independent seekers and discerning users of information in their studies Success Criteria More diverse range of information sources cited in bibliographies Evidence & collection method A team of Learning Advisers to review student bibliographies in partnership with selected academic staff

  22. An exampleInformation literacy framework Objective To raise awareness of the importance of information literacy amongst students and academic staff Success Criteria Every member of academic staff aware of the Information Literacy Framework Evidence & collection method Learning Advisers to carry out personal or telephone interviews to assess level of awareness

  23. An exampleInformation literacy framework Objective To integrate information literacy into the student curriculum Success Criteria Evidence that information literacy is increasingly integrated within the curriculum Evidence & collection method To use an Information Literacy teaching audit to assess current level of integration

  24. Methods used to collect evidence in the LIRG/SCONUL Impact Initiative Diaries Web page hits Face-to-face Interviews Focus Groups Analysis of coursework or bibliographies Telephone Interviews Analysis of records Use statistics [incl. e-measures] Observation & recording of of behaviour Evidence of reflection in progress files Analysis of enquiries Questionnaires Diagnostic tests & assessment

  25. What did participants in the LIRG/SCONUL Impact Initiative find? Birkbeck Glasgow Caledonian Gloucestershire Northumbria Teesside Leeds Stirling Leeds Met UWE Chester

  26. What did participants in the LIRG/SCONUL Impact Initiative achieve? Deeper understanding of how the Library supports academic processes Improved dialogue with academics and stakeholders Supported the management of change Contributed to the development of library staff Raised the profile of the Library within the organisation

  27. Issues identifiedin the LIRG/SCONUL Impact Initiative Support Time Sharing Experience Survey methods Spelling out

  28. More information SCONUL web site http://www.sconul.ac.uk/activities/performance/impact.html LIRG web site http://www.cilip.org.uk/specialinterestgroups/bysubject/research Special issue of Library and Information Research [no. 91, Spring 2005] http://www.cilip.org.uk/specialinterestgroups/bysubject/research/publications/journal/archive/lir91/

  29. More information eVALUEd toolkit http://www.evalued.uce.ac.uk/ Roswitha Poll. Bibliography: impact and outcome of libraries http://www.ulb.uni-muenster.de/projekte/outcome/downloads/bibliography-impact+outcome.pdf

  30. Library and Information Research Group Interested in joining LIRG? Interested in subscribing to Library and Information Research? Contact: LIRG Administrator j.horsfall@leedsmet.ac.uk

  31. Contact details Philip Payne Librarian Birkbeck +44 207 7631 6250 p.payne@bbk.ac.uk

More Related