1 / 61

Using large information and citation databases for evaluation

Using large information and citation databases for evaluation. Tefko Saracevic, PhD School of Communication, Information and Library Studies Rutgers University, USA tefko@scils.rutgers.edu http://www.scils.rutgers.edu/~tefko. Full disclosure. I have no connection with Scopus

adin
Download Presentation

Using large information and citation databases for evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using large information and citation databases for evaluation Tefko Saracevic, PhD School of Communication, Information and Library Studies Rutgers University, USA tefko@scils.rutgers.edu http://www.scils.rutgers.edu/~tefko

  2. Full disclosure • I have no connection with Scopus • But: I am on Scopus Advisory Board & as such have a free password • but I have Scopus access through Rutgers University Library and as Elsevier journal editor • I participated so far at one Scopus Advisory Board meeting (Budapest) and evaluated their product informally over phone conversations • I gave an informal talk about using Scopus at 2006 American Library Association meeting & at Rutgers © Tefko Saracevic

  3. What you can’t find on Scopus Named after: Chiffchaff (Phylloscopus Collybita) a small bird with great navigational skills © Tefko Saracevic

  4. © Tefko Saracevic

  5. Definition of the central theme to evaluate (verb) to consider or examine something in order to judge its value, quality, importance, extent, condition, or performance © Tefko Saracevic

  6. However … • Evaluation has many components and should use a number of sources • Information & citation databases are a powerful source & tool, but one among a number of others • Very useful • But use with skill & caution! © Tefko Saracevic

  7. Overview of Scopus • Elsevier effort to get into searching • & combining ScienceDirect & Scirus (web searching) • Massive effort & outlay; big marketing • development investment HUGE & undisclosed • Headed by Eefka Smit & a young, mostly Dutch team • global operations: • Headquarters: Amsterdam; marketing: global; indexing: Philippines; computers: Dayton, Ohio, USA • Unveiled in 2004 • new features unveiled constantly – innovative • e.g. mid 2005: added RefWorks; end 2005 Citation tracking; 2006 Author profiling & further analysis tools • Search engine licensed from Fast © Tefko Saracevic

  8. Coverage • Science & technology only, no (or little) humanities • includes Chemistry, Physics, Mathematics, Engineering, Life and Health Sciences, Social Sciences, Psychology, Economics, Biological, Agricultural and Environmental Sciences • Covers some 15,000 journals, 700 proceedings, 600 trade publications, 125 book series, 12.5 mill. patents • Incorporates wall to wall Medline, Embase, Compendex, & many other databases © Tefko Saracevic

  9. Coverage … • Time covered: • Abstracts go back to 1966 • References go back to 1996 • While having gaps, coverage seems more comprehensive than any other single database • Also incorporates web search via Scirus • 200 mill. web sources • Also strong in non-English & developing country sources • More than 60% of titles are from countries other than the US © Tefko Saracevic

  10. Overview of other databases- for a few comparisons • Web of Science (WoS) • Coverage: science, technology, humanities • origin in three citation databases • Science Citation Index (SCI), Social Science Citation Index (SSCI), Arts & Humanities Citation Index (AHCI) • at Rutgers coverage only 1994-present - pricing reason - with some 8,000 journals, plus patents & other databases – only this accessible to me • DIALOG • a very large supermarket – some 900 databases (db) in every field and area, including citation indexes • Citation db coverage: SCI 1974- ; SSCI 1972 -; A&H, 1980- • all accessible to me © Tefko Saracevic

  11. Reviews • Comparing Scopus and Web of Science • 2005: http://www.charlestonco.com/comp.cfm?id=43 • 2006: http://www.charlestonco.com/comp.cfm?id=43 • critical of Scopus gaps in coverage, particularly before 1996 • but not clear why comparison of these two services • Scopus does many different things that WoS does not & vice versa • both have citation searching but Scopus has much more • Scopus subject searching is much more comprehensive, WoS citation searching is more comprehensive, but Scopus citation tracking more usable for evaluation © Tefko Saracevic

  12. What can you do? • Subjects search • with many capabilities to limit & modify, rank • Source search – journals, types of sources • Author search with many extensions • – e.g. as to citations to and from • Citation tracking • Integrated with getting full texts with library • Integrated with RefWorks, given library has it • Integrated web search © Tefko Saracevic

  13. What do I do? • Use it as in a variety of roles & evaluations, as a: • researcher • teacher • journal editor • mentor • promotion, tenure, committee member; administrator • tool for keeping current; also: • for finding what and who did I miss • who is leading an area concentrate here with implications © Tefko Saracevic

  14. What do you see? • At first: Lots of features laid out all at once • But, relatively clear interface laying out capabilities • Geared toward fast, intuitive learning & use • and indeed it is relatively easy to learn & use • Results displayed in Last In First Out (LIFO) order, but can be ranked or listed in various ways © Tefko Saracevic

  15. But lets get going …. Live examples from http://www.scopus.com/ user: tsaracevic password: I am not telling or: http://www.libraries.rutgers.edu/ © Tefko Saracevic

  16. Starting … search options © Tefko Saracevic

  17. Use in research and citation tracking • Presently, I have completed but am updating & re-writing a comprehensive review about the notion of relevance in information science • For that: • I did subject searching & identified & evaluated areas of research • I also searched for some key authors and did citation tracking & evaluated contributions & trends • including, of course, a vanity search • then I saved each author or subject search in a list © Tefko Saracevic

  18. Fun part • Had fun tracking those that cited them that cited them … • Eventually got lost in the tracking maze – of course! • Well, lets take a look © Tefko Saracevic

  19. Subject search search selections © Tefko Saracevic

  20. Search results • I found 66 articles about “relevance AND judgment” • then saved them in My List, so I can evaluate, use and update them later • then I found all the citations to the 66 articles • Here is the results page • And then two author examples… © Tefko Saracevic

  21. Searchresults Using options after I got the results © Tefko Saracevic

  22. Following a single author & article • Selected one of the most cited articles: • Saved in list as “Voorhees 2000” and did citation tracking: who cited it? • it was cited 28 times (“Voorhees children”) • then I went on and found 102 articles that cited Voorhees children (“Voorhees grandchildren”) • this way I evaluated impact of an article and spread into various publications and areas • Well, lets take a look © Tefko Saracevic

  23. Selected article various features © Tefko Saracevic

  24. My 11 saved lists after searching& citation tracking I create lists © Tefko Saracevic

  25. Voorhees 2000 I saved in my lists various features © Tefko Saracevic

  26. various features 28 Voorheeschildren © Tefko Saracevic

  27. various features 102 Voorheesgrandchildren © Tefko Saracevic

  28. then… • I selected and viewed the list “Mizzaro citations” to work on them further • selected them all • clicked on citation tracking • and voila! © Tefko Saracevic

  29. Selected them all for citation overview © Tefko Saracevic

  30. Interested in this one © Tefko Saracevic

  31. Follow-up on four articles; Tombros was NEW for me! © Tefko Saracevic

  32. Following a vanity but useful trail • Created a similar list of my own articles • Selected one on interaction & relevance • Who cited it? • Who cited them who cited me? • Discovered a number of previously unknown articles • Well, lets take a look © Tefko Saracevic

  33. Author selection &disambiguation Choice List of all 20 authors last name “Saracevic “– first page © Tefko Saracevic

  34. List of all 5 “Saracevic, T” – all me Author selection &disambiguation List of all 20 authors last name “Saracevic “– second page Choices © Tefko Saracevic

  35. Scopus & I: without self-citations No. of articles in Scopus No. of citations in Scopus This one © Tefko Saracevic

  36. Scopus & I: with self-citations No. of all citations in Scopus 977 all -950 without 27 self © Tefko Saracevic

  37. Web of Science (WoS) • Same subject search “relevance AND judgment” • Same vanity search • Reminder: My access to WoS through Rutgers limited to 1994 – present • Well, lets take a look © Tefko Saracevic

  38. WoS:subject search search selections © Tefko Saracevic

  39. WoS: subject search results search results © Tefko Saracevic

  40. WoS and I: my articles analysis features No. of articles in WoS © Tefko Saracevic

  41. No. of all citations in WoS WoS and I: authors citing me Author citing me most Self citations © Tefko Saracevic

  42. WoS and I: my citations analysis features No. of all citations in WoS © Tefko Saracevic

  43. Dialog • Same vanity search • Reminder: My access to Dialog databases includes whatever years they have: • Citation db coverage: SCI 1974- ; SSCI 1972 -; A&H, 1980- • Dialogweb I use is a command search • powerful but not intuitive at all • needs training or information professional • Well, lets take a look © Tefko Saracevic

  44. Dialog and I: my citations List of databases being searched search command: expand on authors named “saracevic” © Tefko Saracevic

  45. Dialog and I: search process • commands complex, thus screens not shown, except the final result screen • Briefly: • found my articles in all 4 databases (126 articles) • some articles are in more than one db, thus removed duplicates (102 unique articles remained) • found citations to me in all db (1513 citations) • some citations are in more than one db, thus removed duplicates (1084 unique citations remained, but include self citations) • finally, eliminated self citations (1042 citations without self citations) © Tefko Saracevic

  46. S1: no. of articlesin those db S2: no. of articlesafter removing duplicates S3: no. of citationsin those db S4: no. of citationsafter removing duplicates S5: no. of citationsafter removing self citations Dialog and I: search process © Tefko Saracevic

  47. Comparisons of my articles & citations © Tefko Saracevic

  48. Tracking a single article Barry C.L., Schamber L. (1998) Users' criteria for relevance evaluation: A cross-situational comparison Information Processing and Management, 34(2-3), 219-236 • Tracked citations in Scopus • And in Web of Science © Tefko Saracevic

  49. Cited 33 times in Scopus I followed up on the citations – cited even in: Evaluating research for use in practice: What criteria do specialist nurses use? Journal of Advanced Nursing 50 (3), pp. 235-243 © Tefko Saracevic

  50. For Barry & Schamber 1998 article: Scopus: 34 citations Web of Science: 31 citations Oh well … Were they the same articles? Degree of overlap? Overlap: 27 documents (both in Scopus & WoS) Scopus had 7 that WoS did not WoS had 4 that Scopus did not Scopus 34 7 27 4 WoS 31 and the winner is? © Tefko Saracevic

More Related