1 / 45

Here Comes the Sunburst: Measuring and Visualizing Scholarly Impact

Here Comes the Sunburst: Measuring and Visualizing Scholarly Impact. John Barnett Scholarly Communications Librarian Jennifer Chan Assistant Scholarly Communications Librarian Office of Scholarly Communication and Publishing University Library System University of Pittsburgh.

sora
Download Presentation

Here Comes the Sunburst: Measuring and Visualizing Scholarly Impact

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Here Comes the Sunburst:Measuring and Visualizing Scholarly Impact John Barnett Scholarly Communications Librarian Jennifer Chan Assistant Scholarly Communications Librarian Office of Scholarly Communication and Publishing University Library System University of Pittsburgh

  2. Here Comes the Sunburst:Measuring and Visualizing Scholarly Impact

  3. Pittsburgh campus + regional campuses in Bradford, Greensburg, Johnstown, and Titusville • 16 undergraduate, graduate, and professional schools • 456+ degree programs • 2012: conferred 8,949 degrees University of Pittsburgh

  4. Top 10 American higher ed. in federal funding (NSF) • Top 5 in annual research support (NIH) • 5,369 faculty; 4,470 full-time faculty • Research conducted: more than 300 centers, institutes, laboratories, clinics University of Pittsburgh

  5. University Library System • ARL • 22nd largest academic library system in North America • 25 libraries; 6.6 million volumes • 279,000 current serials

  6. Sum of the Parts

  7. Why Pitt? • Strategic goal: Innovation in scholarly communication • Providing services that scholars understand, need, and value • Putting ourselves in faculty “spaces” • Re-envisioning our librarian liaison program • Deepening our understanding of scholarly communications issues

  8. Why PlumX? • Making research “more assessable and accessible” • Gathering information in one place • Making it intelligible and useful • Measuring and visualizing research impact • Correlating metrics from traditional and new forms of scholarly communication • Allowing researchers, labs, departments, institutions to track real-time scholarly impact • Promoting research, comparing with peers, connecting with new research

  9. Altmetrics Project Timeline

  10. Pilot project aims • Develop a tool for measuring and visualizing research impact • Gathering information in one place • Intelligible and useful • Impact in social media and other scholarly communication methods • Traditional measures counted as well • See where to disseminate works to increase impact

  11. Traditional vs. new • Traditional measures are also counted • Findings are complementary to conventional methods of measuring research impact (e.g., H-Index) • Not intended to replace them

  12. New measures • More comprehensive: Altmetrics = ALL METRICS • Citations • Usage • Captures • Mentions • Social Media • Covers impact of online behavior • Because scholars increasingly work online • Measures impact immediately • Because citation counts take years to appear in literature

  13. Pilot Process

  14. Pilot Project Participants • 32 researchers, various disciplines • 9 schools • 18 departments • 1 complete research group • Others joined as they learned about the project

  15. Pilot Project Participants

  16. Technologies • Internal • IR built on Eprints Platform • Sharepoint • Microsoft Office Suite • PMID/DOI data import tool • External • PlumX • DOIs • PMID

  17. Data collection for pilot project • Created records in D-Scholarship@Pitt, our institutional repository • Focused on articles, books, book chapters, proceedings • Scholarly output with standard identifiers • DOI, ISBN, PubMed ID, official URL, etc. • Scholarship produced since 2000

  18. Other Library work • Developed guidelines to standardize record creation • Data entry from faculty CVs into IR (2 to 3 student workers with QA by librarians) • Librarian liaisons and other staff trained in record creation • SharePoint site used to track work completed • Coordination with pilot faculty • Gathered feedback and administered online survey

  19. Sharepoint • Altmetrics Meetings Minutes • Faculty CVs • Excel spreadsheets • Word docs

  20. External Data Sources

  21. Metadata sources • Faculty CVs . . . But verify metadata! • Books: PittCat, WorldCat, Books in Print, publisher sites, online retailers • Journals: Serials Solutions list, journal websites, JournalSeek, UlrichsWeb, DOAJ, PubMed • Conference presentations: Websites, PittCat, indexes, WorldCat

  22. PMID Import Tool • Custom build by SysAdmin for Eprints Platform • Utilizing PMIDs from PubMed, able to import records that prepopulate metadata fields • Item Type, Title, Abstract, Creators, Publication Title, ISSN, Volume/Issue, Page ranges, Date and Date type, DOI, MeSH Headings, Grant Information, Keywords, etc.

  23. Data Ingestion

  24. Full-text sources • DOAJ • ERIC • PLOS • SSRN* • Other repositories* • Federal government websites* • Conference websites* * Use with caution

  25. Plum Analytics processing activities

  26. Key features • Faculty profiles • Online ‘artifacts’ • Article • Book • Book chapter • Video • Etc. • Impact graph • Sunburst

  27. Faculty profile

  28. Online ‘artifact’ display

  29. Impact graph

  30. Sunburst

  31. Feedback • Solicited via email and online survey • Generally positive in most cases • Data corrections • Errors in profiles • Links to wrong data • Quickly corrected by Plum staff • Requests for results from additional online sources (Google Scholar, SlideShare, Reddit, etc.) • PlumX collects data from these but did not gather information in advance for profiles

  32. The survey says • Surveyed pilot project faculty in spring 2013 • @ 1/3rd responded to the survey • Meaning 13 out of 32 participants responded

  33. Accurate and useful data

  34. The bar graph

  35. The sunburst 0

  36. Traditional & new measures 0

  37. Usefulness of altmetrics

  38. Learning something new

  39. Comments • Affiliations/bio inaccurate or has missing information • “Mentions” by whom & when? • Publications misclassified • Books vs. conference proceedings • Data not collected • Google Scholar • Slideshare

  40. Comments • Filter out unwanted information • Data are wrong—and not useful • Overabundance of information in sunburst • “I only care what a select group of scholars thinks of my work” • “I did not find this useful for my discipline”

  41. Observations • Lacked information about faculty practices • Are the results useful to all faculty, all disciplines? • May appeal more to faculty who are early in their careers or whose work is more contemporary • Will the data be used against faculty or programs? • Labor-intensive strategy • When it comes down to it . . . Does anyone care?

  42. Embeddable widgets(in development) For researchers, to add to: • their own Web pages • department directories • IR researcher profile page For individual artifacts,to build article level metrics for imbedding in: • IR document abstract page • Article abstract page for journals we publish

  43. Roll-out challenges • Who creates profiles? Who edits? • What information should be included in profiles? Who can view them? • Separate data gathering from D-Scholarship deposits? • Who promotes the service? Who trains? • Timing . . .

  44. Future plans • Data checking • Additional data gathering • Record merging/deduping • Ability to edit user profiles and artifact records locally • Open API • To allow integration with other online systems • More exhaustive scholarly practices survey for all faculty • Rollout to all Pitt Researchers • Will use automatic feed from Pitt IR to PlumX

  45. Discussion • How would you “sell” PlumXto additional faculty?

More Related