1 / 55

From impact factor to influence?

From impact factor to influence?. Kamran Abbasi Acting editor, BMJ. What I will talk about. What is impact factor? What is wrong with impact factor? What are the alternatives? What is influence? What does all this mean?. Citation rates.

arin
Download Presentation

From impact factor to influence?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From impact factor to influence? Kamran Abbasi Acting editor, BMJ

  2. What I will talk about • What is impact factor? • What is wrong with impact factor? • What are the alternatives? • What is influence? • What does all this mean?

  3. Citation rates • The Institute of Scientific Information records scientific citations/references • The number of times a publication has been cited within a certain period • Published as the Science Citation Index

  4. Impact factor • This is the mean citation rate of all articles contained in the journal • Published annually in the SCI Journal Citation reports

  5. Definition of impact factor • The recorded number of citations within a certain year (eg 2003) to the items published in the journal during the preceding two years (eg 2001 and 2002) • Denominator: An “item” is a research paper or a review article • Numerator: Citations to all types of articles

  6. Calculating the BMJ’s impact factor for 2003

  7. Proper and improper uses of impact factor • Evaluating individual scientists • Awarding higher academic positions • Evaluating research groups • Evaluating institutions • Resource allocation • Evaluating journals

  8. Some examples • In Japan people walk around boasting of their own individual impact factor • In the UK (and elsewhere) university funding is awarded on the basis of impact factors (research assessment exercise) • Deans instruct authors to publish in highest impact factors, though may not be the best fit

  9. What I will talk about • What is impact factor? • What is wrong with impact factor? • What are the alternatives? • What is influence? • What does all this mean?

  10. 1 What is wrong with impact factors? • IF not statistically representative of of individual journal articles • IF correlates poorly with actual citations of individual articles • Authors not only use IF when submitting to journals • The ISI database is imperfect and contains citations to “non-citable” articles • Per Seglen BMJ 1997;314:497

  11. 2 What is wrong with impact factors? • Self citations are allowed • Review articles are heavily cited and inflate the impact factor • Long articles collect many citations • Short publication lag allows short term journal self citations • Same language citations are preferred by authors

  12. 3 What is wrong with impact factors? • Selective journal self citation • Coverage of the database is not complete (3,200 out of 126,000 journals) • Database has an English language bias • Database is dominated by American publications • Journals in database may vary from year to year • IF is a function of the number of articles in the research field

  13. 4 What is wrong with impact factors? • Research fields with literature that rapidly becomes obsolete are favoured • Impact factor depends on expansion or contraction of research field • Small research fields lack journals with high impact • Relations between fields (eg clinical v basic science) strongly determine IF • Citation rate of article determines journal impact, but not vice versa

  14. What I will talk about • What is impact factor? • What is wrong with impact factor? • What are the alternatives? • What is influence? • What does all this mean?

  15. “The combined shortfall of university funding has forced deans of medical schools to behave like managers of Premier Division soccer clubs, recruiting potential stars . . .at the expense of teaching and clinical practice”Banatvala et al Lancet 2005;365:458-9

  16. 1 What are the alternatives? • Will papers from high impact factor journals be cited in evidence based guidance? • Yes there is a correlation but papers from journals with low impact factor are also cited frequently • Nakayama JAMA 2004;290:755-6 • Might citations in evidence based guidelines be a better clinical measure?

  17. 2 What are the alternatives? • A patient citation index • Proposed by Martin Rosser, editor of JNNP, and by Mary Baker and Matthew Menken from BMJ’s patient advisory board • A fuzzy idea but the gist that patients would rate articles of importance to them and hence of greater clinical importance

  18. 3 What are the alternatives? • The Leiden University system • This is a method of measuring academic achievement through monitoring mentions of an article in the media, in parliament, and other such markers of “influence” • It requires a lot of work but may be fairer

  19. 4 What are the alternatives—from rapid responses on bmj.com • User rating of articles • Use a longer term base than 2 years • Exclude letters and reviews to focus on research • “Scope adjusted impact factor” • Expert evaluation of best papers • Many from Steve Harnad around citations, downloads, and google type page ranks • Faculty of 1000 assessment www.f1000.com • Ask clinicians

  20. What I will talk about • What is impact factor? • What is wrong with impact factor? • What are the alternatives? • What is influence? • What does all this mean?

  21. Why do we care about influence? • The mission of the BMJ Publishing Group has two parts • INFLUENCE: To serve the needs of doctors and others, to influence the international debate on health • PROFIT: to make enough money to support the mission of influence

  22. Why do we care about influence? • Profit is easily measured—down to the last penny • We are not quite sure what influence is, which makes it hard to measure • Yet influence is the first part of our mission and profit the second • We mustn’t allow the important to be displaced by the measurable

  23. Definitions of influence • “The power of producing an effect, especially unobtrusively” Chambers dictionary • (Mark Twain said: “If you don’t mind who gets the credit you can do anything.”) • Influence is in some ways a polite word for power. • Influence is also something to do with brand. A stronger brand=more influence.

  24. What is influence? • Level one: something changes because of what we have published • Doctors change what they do. • Ministers change policy. • WHO decides to do things differently. • Drugs are prescribed more or less. • New techniques or methods are adopted. Old ones are abandoned.

  25. What is influence? • Written information on its own rarely leads to change • “All journals do is take in other people’s washing” • This sort of influence is probably rare and is hard to identify. • Many different factors usually contribute to a particular change: so even if something we published contributed it could not be described as the cause.

  26. Examples of change caused by research articles I • Photodynamic therapy with a new drug might cause severe burns • Hettiaratchy S, Clarke J, Taubel J, Besa C. Burns after photodynamic therapy. BMJ 2000; 320: 1245

  27. Examples of change caused by research articles II • The use of albumin in critically ill patients may be dangerous • Cochrane Injuries Group Albumin Reviewers. Human albumin administration in critically ill patients: a systematic review of randomised controlled trials. BMJ 1998; 317: 235-240

  28. Examples of change caused by research articles III • Minocycline should not be used as the first line treatment of acne • Made the front page of the Daily Mail, which might be Britain’s most influential newspaper • Gough A, Chapman S, Wagstaff K, Emery P, Elias E. Minocycline induced autoimmune hepatitis and systemic lupus erythematosus-like syndrome. BMJ 1996;312:169-72

  29. What is influence? • Level two: setting an agenda or legitimising an issue • Examples of where the BMJ might have done this, at least in Britain • Evidence based medicine • Inequalities in health • Prison health care • Medical error • Doctors and the drug industry

  30. What is influence? • Level three: leading by example and being folowed • Possible examples • bmj.com is free • open peer review • BMJ ethics committee • rapid responses on bmj.com • collected resources on bmj.com • BMJ patient advisory group

  31. What is influence? • Level four: being quoted/cited • “The only thing worse than being talked about is not being talked about.” Oscar Wilde

  32. What is influence? • Where might you be quoted? • Other journals (impact factor) • Cochrane reviews • Guidelines • Mass media • Parliament (Hansard) • Evidence Based Medicine/Journalwatch • Important policy documents (for example, Institute of Medicine report) • Presentations

  33. What is influence? • Level five: being paid attention to • Readership (preferably judged by others) • Website hits (in a week or over time) • Sales

  34. What is influence? • Level six: being known about • Widely known even if not quoted or read among international health professionals, political leaders, the public • If George Bush knows about you that’s more influential than if your mum does—sadly

  35. Scoring influence the Richard Smith way • Level one: creating change • Several clear cases 5 points • One case 3 points • Level two: setting agendas and legitimising issues • Several cases 5 points • One case 3 points

More Related