“The coolest thing to do with your data will be thought of by someone else”

1. “The coolest thing to do with your data will be thought of by someone else” Dave Pattern Library Systems Manager University of Huddersfield d.c.pattern@hud.ac.uk www.daveyp.com/blog/ Graham Stone Information Resources Manager University of Huddersfield g.stone@hud.ac.uk @Graham_Stone

2. …to improve existing services …to gain insights into user behaviour …to measure the impact of the library do the students who use the library the most get the highest grades? Using Usage Data…

3. User activity data record of a user’s actions on a website or software system or other relevant institutional service Attention data record of what a user has viewed on a website or software system or other relevant institutional service http://bit.ly/g6S5wH Defining Using UsageJISC Activity Data Programme

4. Circulation Transactions user A borrowed this book on 23/Jan/2011 E-Resource Usage user B logged in and accessed ScienceDirect 126 times during 2009 Entry Stats user C entered the library 12 times in Dec/2009 Defining Usage Datasome library examples...

5. Usage Data at Huddersfield

6. Keyword Cloudbased on keyword search data 6

7. Borrowing Suggestionsbased on circulation data

8. Borrowing Suggestionsbased on user’s recent loans

9. Course Level New Book Listsanalysis of course loans & Dewey

10. Search Suggestionsbased on keyword search data 10

11. Guided Keyword Searchesbased on keyword search data 11

12. Journal Suggestionsbased on link resolver logs 12

13. The Impact of Serendipity

14. Trends in Borrowingunique titles circ’ing per acad. year 14

15. 17.10 18.92 Trends in Borrowing# of items borrowed per acad. year 15

16. Discussing Open DataJISC TILE Project meeting (Jun 2008) 16

17. “The coolest thing to do with your data will be thought of by someone else.” Rufus Pollack (Open Knowledge Foundation) http://m.okfn.org/files/talks/xtech_2007/ Sharing DataWhy should I release my data?

18. “Absolutely, couldn’t agree more. The point is not so much whether this statement might be true or not, so much as what it does to your thinking and planning if you decide to take it as an article of faith.” Paul Walk (UKOLN) http://bit.ly/9ao3c4 Sharing DataWhy should I release my data?

19. Sharing Datausage data release (Dec 2008) 19

20. Sharing DataBA Multimedia Design 20

24. Non & Low Usage Project

25. Non & Low Usage Projectdigging deeper into data since 2005! 25

26. Non & Low Use Projectdigging deeper into data

27. Measuring Library Impactis there a link between usage & grade?

28. Non & Low Use ProjectResults • Not a cause and effect relationship • Never proven statistically significant • Potential for collaboration on future projects http://www.flickr.com/photos/atoach/3344411469/

29. JISC Activity Data Call

30. Library Impact Data Project

31. To prove the hypothesis that… “There is a statistically significant correlation across a number of universities between library activity data and student attainment”

32. Project reportshttp://library.hud.ac.uk/blogs/projects/lidp/ • Themed posts • The Project Plan • Hypothesis • Users • Benefits • Technical and Standards • Licensing & reuse of software and data • Wins and fails (lessons along the way) • Final post

33. Data requirements • For each student who graduated in a given year, the following data was required: • Final grade achieved • Number of books borrowed • Number of times e-resources were accessed • Number of times each student entered the library, e.g. via a turnstile system that requires identity card access • School/Faculty

34. Legal issues • Consultation with JISC Legal, University legal officer and data protection officer • Ensured that any identifying information is excluded before it is handled for analysis • Excluded any small courses to prevent identification of individuals e.g. where a course has less than 35 students and/or fewer than 5 of a specific degree level • Received guidance from the Using OpenURL Activity Data

35. Data issues • Anticipated that there may be problems in getting enough data to make the project viable • Potential partners were asked to confirm that they could provide at least 2 of the 3 measures of usage as well as student grades • Huddersfield has provided definitions on the data required and the form the data can be accepted in • Some partners have already run into some issues with data collection, but it is felt that there is still enough information to prove the hypothesis one way or another

36. Can we prove the hypothesis? • Not quite! • Due to the data not being continuous, a correlation cannot be calculated http://www.flickr.com/photos/26015375@N06/3715306069/

37. Further statistical tests • Running a Kruskal-Wallis test • to indicate whether there is a difference between values e.g. between levels of e-resource usage across degree results • THEN we analyse the data visually to check which variables to compare

38. What we think we can prove • That the relationship and variance means that you can believe what you see • And you can believe it across a range of data, e.g. subjects • So library usage does impact on students attainment

39. Remember the disclaimer! Not a cause and effect relationship

40. Library Impact Data Projectbook loans inc. renewals (2009/10)

41. Library Impact Data Projectbook loans & Athens (2009/10)

42. Library Impact Data Projectlibrary PC logins & visits (2009/10)

43. Linking back to non/low usage • Our research shows that for books and e-resource usage, there appears to be a statistical significance across all partner libraries • If we know that there is a link between usage and attainment • We can link this back to non/low usage

44. Measuring Library Impact2008/9 – library visits 15.5% of students who gaineda 1st never visited the library 34% of students who gaineda 3rd never visited the library 44

45. Measuring Library Impact2008/9 – MetaLib usage 10.5% of students who gained a1st logged in more than 180 times 70% of those who gained a3rd logged in to e-resources20 times or less over 3 years 45

46. Measuring Library Impact2008/9 – book loans 15% of students who gaineda 1st never borrowed a book 34% of students who gaineda 3rd never borrowed a book 46

47. Profiling non/low users • Flesh out themes from the focus groups • to advise on areas to work on • Check the amount and type of contact subject teams have had with the specific courses • to compare library teaching hours to attainment • Baseline questionnaire or exercise for new students • To establish the level of information literacy skills for new students • Target our users by concentrating staff resources at the right point

48. Next steps for the project Pull out any themes from the focus groups Release the data on an Open Data Commons Licence Release a toolkit to help others benchmark their data

49. Further work • Do cuts to the information budget mean that attainment will fall? • Can we add more value by better use of resources? • What about gender and socio-economic background? • Can we link books borrowed on reading lists to attainment • What about VLE use? • Interest from ACRL (USA), University of Wollongong (Australia), the Netherlands and Denmark

50. Acknowledgements Dave Pattern and Bryony Ramsden Phil Adams, Leo Appleton, Iain Baird, Polly Dawes, Regina Ferguson, Pia Krogh, Marie Letzgus, Dominic Marsh, HabbyMatharoo, Kate Newell, Sarah Robbins, Paul Stainthorp