Resolvability of References in Users’ Personal Collections

1. Resolvability of References in Users’ Personal Collections Nishikant Kapoor, John T Butler, Gary C Fouty, James A Stemper, Joseph A Konstan GroupLens Research Group and University Libraries University of Minnesota

2. Personal Collections • Class projects • Students, faculty • Research papers, proposals, workshops • Academics, industry • Collaborate with fellow researchers • Within and across groups • Intra and inter disciplinary

3. User Profile • Users’ online personal reference collections • Represent users’ profile • Research interests • Research collaborations • Single collections • Diverse • Multiple collections • Task based • Workgroup based

4. Research Objectives • Understand • Users’ personal collections • Define • Users’ profile • Develop • Personalized digital library services such as reference recommender system

5. Research Questions • How can we utilize users’ personal collections to offer them personalized services? • Can references in users’ personal collections be resolved to unique identifiers? • How many of those do actually resolve to a unique online identifier? • How can we enhance resolvability of references in these collections?

6. Our Approach • Most research work focuses on mining • Textual content of references • Implicitly rated citations in the reference section • Our focus is users’ personal collections • Identifying them to a unique ID • Use these collections as an implicit means to define users’ profile

7. Data Collection • RefWorks users • 96 collections • 30,336 references • Two outliers (4000+ and 7000+)

8. Resolvability • A reference is resolvable if it has • A valid unique ID that leads to its online source: DOI, URL, ISBN • URL may or may not be unique • Enough information to resolve it to its online source • All references that can be represented using a valid unique ID, are potentially resolvable (assuming they exist online).

9. Total potentially resolvable references: 24,719 + 548 + 74 + 2,216 = 27,557 (90.84%) Potential Resolvability

10. Refs With Actual IDs

11. Actual Resolvability 3,765+793+622 = 5,180 (17.07%) 1,932+771+544 = 3,247 (10.70%)

12. External Resolvers • DOI and OpenURL Query Interfaces • Reference resolver at crossref.org • ISBN Query Interfaces • Reference resolver at worldcat.org • URL Validation • Validated existence of URL, not its accuracy • Did not attempt to retrieve reference ID

13. Resolvability Computation Resolvability = 60+1+8646+213+321+1349+1914+9 = 12,513 (41.25%)

14. Concerns • Not all intellectual content is digitized yet • If an ID could not be retrieved programmatically, we assumed the reference is un-resolvable. • Using additional reference resolvers could enhance resolvability further • Privacy concerns • Users’ private collections MUST remain private

15. Limitations • Dataset • Too diverse to form meaningful correlations among users • Very small to develop CF based DL services

16. Summary and Results • Potential resolvability • Over 90% of the total references in users’ personal collections could possibly have a valid ID (DOI, ISBN, URL) • Available resolvability • About 17% references had a valid ID • Actual resolvability • Fewer than 11% references resolved successfully • Enhanced resolvability • Over 41% of total references could be resolved

17. Future Work • Understand how users maintain their personal reference collections • What tools and methods do they use • Import, hand-typed, cut ‘n’ paste • Understand how truly do these collections represent users’ profile? • What degree of reference overlap can we expect among the collections of different users?

18. Thanks! • NSF • RefWorks (http://www.refworks.cm/)

19. Resolvability of References in Users’ Personal Collections Nishikant Kapoor Questions?