1 / 22

Usage metrics & Usage concerns

Usage metrics & Usage concerns. Rupert Gatti. Usage metrics: Uses. At OBP we have used usage metrics to demonstrate: The high levels of engagement with our content The geographic reach of our published content Differences in engagement/popularity by format

mvallecillo
Download Presentation

Usage metrics & Usage concerns

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Usage metrics & Usage concerns Rupert Gatti

  2. Usage metrics: Uses At OBP we have used usage metrics to demonstrate: • The high levels of engagement with our content • The geographic reach of our published content • Differences in engagement/popularity by format • Engagement with publications over time • University usage for reports to libraries But we have reservations for using it to actually measure anything …

  3. https://reports.openbookpublishers.com/public/report/10.11647/OBP.0125https://reports.openbookpublishers.com/public/report/10.11647/OBP.0125

  4. Ethics for A-level: Lessons • Accessed over 23k times – in just over a year • More users from the US than the UK • Online users seem to mildly prefer the html edition to the pdf reader. • Book downloads are about 15% of online usage • Usage highest during school/uni term, and higher this academic year than last academic year We have made some effort to market this to first year undergraduate courses in the US (as well as to A-level students in the UK) – so we find this type of data useful in evaluating the effectiveness of our efforts.

  5. https://reports.openbookpublishers.com/public/report/10.11647/OBP.0025https://reports.openbookpublishers.com/public/report/10.11647/OBP.0025

  6. Oral Literature in Africa: Lessons • Over 170k online users • Over 60% of online users come from Africa (Kenya, Nigeria, Tanzania) • Online readership has not tailed off in the six year since publication - but book downloads seem to have peaked in 2014 • On the OBP website – relative popularity of HTML and PDF readers has varied over time • Open Edition is becoming an increasingly important discovery/access platform for online users, while Google Books less important • Free downloads from retail distributors (amazon & google play) have been significant.

  7. Library usage reports OBP has a library membership scheme, and as part of this provides annual usage reports for the university Which they compare with ebook usage reports for non-OA titles libraries want COUNTER compliant reports we need to aggregate usage figures across types OBP only has data for usage on the OBP site so ‘missing’ usage data from titles hosted on all other sites

  8. DANGER – numbers give a false sense of accuracy Just defining a usage/download is download • Huge numbers of bots, which are screened out of the statistics • Defining a “use” • what is a “session”? • what if somebody clicks the download button twice? We have three different ways of measuring usage on our website – Google Analytics, Matomo and direct analysis of our weblogs (COUNTER) all three give us different numbers! Which should we use? (we report GA on website, COUNTER to libraries)

  9. GA v COUNTER: 25% difference

  10. Downloads vs Online Readers • Can I just add these together? • Some sites allow you to do either (eg OBP) – so you can browse before downloading • Some sites only allow downloads (eg OAPEN) – so you need to download to see anything • How different are uses and users (do users come back often to read/download from the same platform)? • Downloaded titles can be shared … how do I assess those? • Will I ever know the number of uses of a downloaded title? • Actually I could monitor that – but should I?

  11. Volume vs Chapter data • Many sites only allow chapter downloads (eg JSTOR), others only whole book downloads (eg OAPEN), and some allow both (eg OBP) • How do you combine these different types of data? • In some cases session data is available (OBP) – but often not (JSTOR) • OBP only reports session data (so presently not reporting JSTOR) • With chapter level data - readers wanting the whole book generate more downloads if the book has more chapters Ref: my blogpost “Handle with Care: pitfalls in analysing book usage data”

  12. https://www.springernature.com/gp/open-research/journals-books/books/the-oa-effecthttps://www.springernature.com/gp/open-research/journals-books/books/the-oa-effect

  13. All three effects are potentially bogus! But I’ll just look at the downloads figure now: • SN aggregates everything into chapter downloads – so if you download a single pdf of the entire book (with, say, 20 chapters) it is recorded as 20 ‘chapter’ downloads. • Thus – if users faced with buying content prefer to by a single chapter to the entire book, but prefer to download the entire book rather than a single chapter when both a free – then a transition for non-OA to OA should increase the number of recorded downloads by 20, even thought the number of users is unchanged. • (of course the user now has access to more content, but … is that what we are counting?) “Handle with Care: pitfalls in analysing book usage data” https://rupertgatti.wordpress.com/

  14. Where is the reference (DoI) landing page? At OBP we use the ‘book page’ as the reference url for our titles. Readers then receive quite a lot of info about the book, and then can make a choice about if/how they would like to access the full content: download/online pdf reader/html/xml or purchased copy We think this is the most useful place for a reader to land. But we would have many more download/usage records if we chose one of the content pages as the landing page.

  15. We are missing lots of data … • How much? – we don’t know • But we do know our books can be downloaded from many sites where we don’t get any usage reports • Author homepages, academia.edu, research gate, university repositories …. • Which is great for disseminating the book – but problematic if the usage stats ‘matter’ to either us or the author • Geographical data even worse … HIRMEOS is directly trying to reduce these problems

  16. Gaming the stats …. • Of course – if anything important depends on the usage stats reported (for author or publisher), then there are incentives to change behaviour • Relink DoIurls • Redefine session lengths • Discourage posting/sharing of content on sites with no usage data • Be less rigorous in cleaning bots • Create bots to inflate usage data …..

  17. Social media data is even worse! We observe that the majority of ‘tweets ’ about OBP books do not include a url to a bookpage.

More Related