1 / 44

Journal Selection and Journal Metrics.

Journal Selection and Journal Metrics. . Journal Impact, the Impact Factor and the results of international expansion. Jeffrey Clovis Senior Director, Thomson Reuters Jeff.Clovis@thomsonreuters.com September 2012. Overview. The Foundation Building reliable metrics

lieu
Download Presentation

Journal Selection and Journal Metrics.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Journal Selection and Journal Metrics. Journal Impact, the Impact Factor and the results of international expansion • Jeffrey ClovisSenior Director, Thomson Reuters • Jeff.Clovis@thomsonreuters.com • September 2012

  2. Overview • The Foundation • Building reliable metrics • Creating a citation index – how article indexing becomes journal metrics • Content of the JCR – Overview; Citation network; ranking metrics • Understanding the Journal Impact Factor • 2008-2010 Coverage expansion and regional journals • Rules for use of citation metrics • Other Journal Metric Resources (JPI)

  3. The Data Foundation In 1955, Dr. Eugene Garfield revolutionized research with his concept of citation indexing and searching, creating the Science Citation Index® Thomson Reuters introduced the first ever research evaluation tool on the Web Web of Science is the largest browsable, citation database with over 1 billion cited references from 1898 to 2012

  4. Building Reliable metrics :Journal Selection Process Journal Publishing Standards International Diversity: Authors, EAB Editorial Content Citation Analysis • Will this journal provide novel content? • •Is this subject already well covered? • •How does this journal compare with covered journals of similar scope? • Regional specialization • Timeliness of publication • International Editorial Conventions • English language Bibliographic Information • Peer Review • Do authors, editors, EAB members represent the int’l research community? • Does this journal target an International or Regional audience? • New journals: • Citations to authors’, editors’ prior work. • Examination of Editorial Board • Established Journals: • Impact Factor

  5. Disparate Data in ManyFormats

  6. Citation Indexing • Definition: a relational database of metadata about Source items and metadata about cited references Cited reference index Source index Cited Ref A (1) Cited Ref B (2) Cited Ref C (2) Cited Ref D (1) Cited Ref E (1) Article 1 Article 2

  7. Building metrics • Selectivity = Authority • Known, trusted source of cited references • Defined, controlled, consistent coverage • Established rules for processing of over 50 different reference types (Journals, Meetings, Books, Books in Series, Technical Reports, Special Publications, and hierarchies for processing when any of these are presented together). • Indexing = Accuracy • Technology and curation to overcome variations in original full text • 50 Years experience in the development of processes to handle FT variations • Data structure overcomes boundaries of coverage

  8. Journal Citation Reports

  9. 2005 2006 2007 2008 2009 Impact Factor – Great Strength in Its Simplicity 2008 “Impact Factor Years” Citations captured from Web of Science journals during the IF publication year to any/all items published within a journal in the prior two years. ____________________divided by___________________ The number of Articles and Reviews (citable items) published in a journal in the prior two years. Impact Factor is Transparent, very Easy to Understand

  10. Indexing – from journal to database Cited Reference indexing BP Source item indexing

  11. Indexing – from database to product Web of Science Link citations to source articles Cited Reference indexing BP Cited Work Cited Volume Cited Year Aggregate citations to journal Citation data (IF Numerator) JCR Source data (IF Denominator) - Publication Year - Document Type Source item indexing All Bibliographic fields Incl: Document type Web of Science CONFIDENTIAL

  12. Content of the JCR

  13. Some terminology – and a visual convention Cited Citing Time

  14. Incoming Citations Outgoing Citations Source indexing Content of the JCR: Three kinds of data • Performance metrics (ranking) • Impact Metrics - Frequency of citation • Total Citations • Immediacy Index • Impact Factor (JIF and 5 Year IF) • Cited Half Life • Citing Half Life • Influence metrics – Citation network analysis • Eigenfactor Score™ • Article Influence Score™ • Citation Network (descriptive – space and time) • Cited journal table • Citing Journal table • Related Journals • Publication information (descriptive - content) • Subject category • Source data

  15. Citation Relationships Citing Citing Cited Cited Cited journal: Tabular display of all journals citing the target journal; shows both number of citations and year cited Citing journal: Tabular display of all journals that were cited by the target journal; shows both number of citations and year cited

  16. Impact Factor Imm. Index Total Cites Cited Journal list Eigenfactor & Art. Inf. Journal Self-Citations Cited Half-life = 8.2 yrs

  17. The ranking metrics

  18. Source paper – published in 2010 Cited reference – to paper published in 2005-2009 Cited reference – published in 2008 or 2009 Two key ranking metrics – impact and influence Journal Impact Factor Time All Prior Years 2005 2006 2007 2008 2009 2010 2011 Eigenfactor™ metrics Citations

  19. Impact Metrics Journal A 1000 citations Journal B 1000 citations 100 citable items 1000 citable items JIF=10.000 JIF=1.000 Simple ratio – Citations : citable items

  20. Influence metrics – Citation network structure • Current year citations to 5 prior years from Cited journal data tables (Excluding journal self-citations) • Some metaphors – random walk; Percentages Eigenfactor™ Score and Article Influence™ Score

  21. Global Expansion of coverage1998-2009

  22. Regional Content Expansion • Mission: • Meet the needs of the emerging and growing regional community of Web of Science users. • Identify, evaluate, and select the best Regional Journals. • Regional journals target a more regional rather than an international audience. • Regional journals focus on topics of regional importance (e.g. social sciences, clinical medicine, agriculture) • Cover Regional Journals that complement our renowned international coverage in all areas of science, social science, and arts & humanities. Expand the Editorial Scope of Web of Science to include the best international and regional journals published today.

  23. Web of Science GrowthFocus: 5 Countries within Region- LA

  24. Expansion of Web of Science to reflect regional specialization • 2010 onward • Observe changing citation dynamics among Web of Science journals as a result of increased coverage Regional Journals • Refine selection criteria for Regional Journals incorporating rigorous contextualized citation analyses (category, region, country etc.)

  25. Latin American Titles in the JCR

  26. Unique cited works The complete set of 534,957 cited refs contained 136,478 unique cited works The titles added due to the Regional Expansion contained 86,693 cited works that were unique compared to the Non-REx population Increasing coverage from a country results in increased ability of the citation index to detect materials from that country

  27. Building Impact

  28. Building Impact – right ways Chew, Villaneuva and Van Der Weyden (2007) Journal of the Royal Society of Medicine • Surveyed current and former editors of top medical journals for factors that affect the Impact Factor: • Active recruitment of high-impact articles by courting researchers, expanding Editorial staff • Offering better author services • Boosting the journal’s media exposure • More careful article selection • The best way to improve a journals’ impact factor is to ensure the highest quality publication possible, both scientifically in journal management

  29. How use of the Journal Impact Factor becomes abuse • Impact Factor, long used as a measure of a journal’s impact on the scholarly literature, becomes a proxy for the determination of scientific value • IF, a journal-level measure, is applied to the evaluation of researchers, and their research • Researchers feel pressure to demonstrate their contribution by publishing in a journal with a high Impact Factor • IF becomes a motive force in the selection of publication venue • Publishers/editors, to attract the best possible contributions, feel pressure to demonstrate a high Impact Factor. • IF becomes a consideration in editorial and publication practices. • Misuse of the metric results in attempts to manipulate …

  30. Building “impact” the wrong way:Some recent examples • REV BRAS FARMACOGN: First Journal Impact Factor in 2009=3.462 • Journals whose rank in category is significantly distorted by self-citation are removed from JCR for 2 years, then re-evaluated Journal self-citation - 2009

  31. Citation concentration – time Journal self-citations are concentrated in Journal Impact Factor years High-value citation partners show extreme concentration

  32. Citation concentration – space

  33. Understanding Impact

  34. What an Impact Factor IS: • A journal level metric: normalizes citation count by the amount of scholarly, citable content What an Impact Factor is NOT: • A metric of the quality of an individual’s, department’s, or institution’s scientific contribution • An article level metric. Individual article citation counts vary greatly even within a single volume and year

  35. Time Citation What an Impact Factor IS: • Reactive • Reflects actual citation use of articles two or three years after their initial publication • Because the IF uses the two most recent years of cited material, changes (for good or ill) can be rapidly reflected in changes in the IF What an Impact Factor is NOT: • Predictive • Citation looks backward in time • Impact Factor follows citation

  36. What an Impact Factor IS: • Relative and comparative • Publication and citation practices differ by discipline • Impact Factors should only be compared within a category • Category-level citation data provide a context for understanding journal performance What an Impact Factor is NOT: • Absolute • IF should not be compared across fields as a measure of relative quality of different subjects

  37. Rules for Use of Citation Data • Consider whether available data can address question • Compare like with like • Use relative measures, not just absolute counts • Recognize skewed nature of citation data • Confirm data collected are relevant to question • Ask whether the results are reasonable Using Bibliometrics in Evaluating Research http://scientific.thomsonreuters.com/bibliometrics2/

  38. Some other Journal and Category Metrics

  39. Journal Performance Indicators (JPI) • JPI is a database used for tracking journal performance over the last 31 years (1981-FORWARD), and for comparing the performance of different journals. • covers over 12,5000 of the world's most-cited journals indexed in the Web of Science (including science, social science, and arts/humanities journals), and has a user friendly interface • provides statistics on number of papers, number of citations, citation impact (average cites per paper), impact relative to worldwide field averages, percent papers in a journal category, etc. • JPI is a great addition to your other collection development tools. It offers a range of different citation measurements, enables trend analysis, and covers all journals indexed in Web of Science.

  40. Analytical options • Here are your options for analysis • 1. Level of analysis: journals or journals compared to their category • 2. Time period: You can look at single periods like the last 5 years, or trends like 5 years windows. • Type of analysis: papers, impact (cites/papers), relative impact, etc.

  41. Impact relative to category Rel Imp is the ratio of the impact of the journal to the field impact base. A relative impact above 1.0 indicates the journal is performing above the average for all journals in its category. Impact is the average cites per paper for the journal in this time period. Imp Base is the field average impact. Environment/ Ecology papers published between 2002 and 2006 on average received 3.76 cites from 2002-2006.

  42. Contextualization of citation counts, benchmarking Small number of Publications/Citations, but strong impact Discover influential, emerging researchers & trends Identify areas for growth and investment

  43. Thank you! • http://community.thomsonreuters.com/t5/Citation-Impact-Center/bg-p/citation • http://wokinfo.com/products_tools/analytical/jcr/ • ts.production.tsagjcr@thomsonreuters.com

More Related