210 likes | 312 Views
Objective Collections Evaluation Using Statistics at the MIT Libraries. Mathew Willmott MIT Libraries ACS National Meeting and Exposition August 22, 2010. Overview. Introduction/Background Data Gathering Data Analysis Decision Process Applications Future Work. Introduction: Statistics.
E N D
Objective Collections Evaluation Using Statistics at the MIT Libraries Mathew Willmott MIT Libraries ACS National Meeting and Exposition August 22, 2010
Overview • Introduction/Background • Data Gathering • Data Analysis • Decision Process • Applications • Future Work
Introduction: Statistics • “There are three kinds of lies…” • Shortcomings of anecdotal evidence • New technology for dissemination enables new technology for evaluation
Introduction: Financial Issues • In the world • At MIT • In the MIT Libraries
Introduction: Library Collection • Size of collection • Focus of collection • Cancellation project feasibility
Data Gathering: What data? • Cost • Usage • Impact Factor/Subject ranking • Papers published by MIT researchers • MIT-affiliated editors • Citations from MIT-authored papers
Data Gathering: From where? • Our budget commitments database • Publisher-distributed reports • Journal Citation Reports • Licensed databases • Journal web pages • Local Journal Utilization Report
Data Gathering: How? • Mostly manual • Some selective • Small team gathering for all librarians
Data Analysis Based analysis on four main data categories: • Cost per use • Average subject ranking • Papers published by MIT researchers • Presence of MIT-affiliated editors
Data Analysis • Ranked journals in each category of data • Assigned a “point” to the lowest performing journals in each category: • Lowest 50% by cost per use • Lowest 33% by subject ranking • Lowest 50% by papers published • No MIT-affiliated editors • Each journal ended up with a score of 0 (high-performing) to 4 (low-performing)
Data Analysis Data presented to librarian staff in Excel workbook: • All raw data • Sheets analyzing each category of data • Sheet assigning a score to each journal, with changeable criteria
Example of spreadsheet Lowest 50%: Cost per use > $20
Example of spreadsheet Lowest 50%: Cost per use > $20 Lowest 20%: Cost per use > $50
Example of spreadsheet Change the $20 per use criteria value…
Example of spreadsheet …to a $50 per use criteria value.
Decision Process • NOT used to make final cancellation decisions; important to note that there are other factors to be taken into account. • Used to identify candidates for cancellation that subject librarians would then examine more carefully.
Applications • Faculty and other stakeholders are very data-driven; this process allows for clearer explanations and communications • Process encourages a big picture view across all disciplines • There are some caveats: can’t cancel much from one publisher, society packages aren’t comparable…
Future Work: Other data • Trends from year to year • Eigenfactor/Article Influence Score • More LJUR data
Future Work Can be of use when not in cancellation mode: • Evaluate collections • Identify where money could be better spent • Identify which parts of the collection need better promotion
Thank you! Contact: willmott@mit.edu (photo credit: Flickr user neilio)