A New Measure of Knowledge Diffusion

1. A New Measure of Knowledge Diffusion Stephen Carley; Alan Porter Georgia Tech

2. Our Interests • Interest in research interdisciplinarity • Possible key to more creative research • Possible means to solve complex scholarly and societal problems • Interest in research knowledge diffusion • Trans-disciplinary knowledge enrichment • Translation of research to clinical practice • Fostering science-based innovation

3. Measures • Have devised metrics: • Integration score [how diverse are the research knowledge resources upon which an article (or body of research) draws?] • Specialization score[how diverse are the venues in which a body of research (e.g., the works of one author in one time period) appears?] • Now adding -- Diffusion score [how diverse are the fields that cite an article (or body of research)?]

4. Heuristics of diversity(Stirling, 1998; 2007)(Rafols and Meyer, 2009) [** Shannon & Herfindahl do not include Disparity] Diversity: Attribute of a system whose elements may be apportioned into categories Characteristics: Variety: Number of distinctive categories Balance: Evenness of the distribution Disparity: Degree to which the categories are different. Variety Shannon (Entropy): i piln pi Herfindahl (concentration):  i pi2 Dissimilarity: i di Balance Disparity Generalised Diversity (Stirling) ij(ij) (pipj)a (dij)b

5. Integration where i = row, j = column, f = frequency, SC = Cited Web of Science Subject Category;cosine similarity of SC cross-citation (based on one year of Web of Science journal cross-citation) Integration Defined: equivalently: Integration Range: 0 → 1 Integration Benchmarking: ~0.42 for much of modern scientific research (increasing moderately)

6. Changes in Average Integration Scores over Time • Porter, A.L., and Rafols, I. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.

7. Diffusion Scoring Example Paper 1 Paper 2 Cited by: Biology Genetics Math Music Forestry • In the example above Paper 2 will likely have a higher diffusion score than Paper 1 because there is greater heterogeneity among its citing subject categories [variety & disparity; not clear on balance].

8. Diffusion Scoring • • Diffusion Defined: • where: • pi = proportion of citing references corresponding to the SC i in a given paper • sij = cosine measure of similarity between SCs i & j for cross-citation in WOS • Diffusion Range: 0 → 1 • Diffusion Benchmarking: ~0.42 for much of modern scientific research [but somewhat higher for papers “out there” longer]

9. Mean Diffusion scores for all benchmark years combined Modest increase over time since publication

10. Diffusion Score Behavior • Diffusion score behavior is more challenging to study than Integration: • Calculating Diffusion score requires additional “citation searching” in Web of Science to retrieve the citing papers • A paper’s Diffusion score is not fixed; over time it will accrue more citations • Self-citation could affect Diffusion score • We examined – for 5 of our 6 target Subject Categories -- minimal effect • For Math – removing self-cites increased scores for 1995 publications by 0.01-.02, until recent years [Since Math 1995 annual Diffusion scores are ~0.22, this is ~5-10% increase]

11. Self-Cites (1)

12. Self-Cites (2)

13. Diffusion Scores: Total vs. Annual • Annual Diffusion may be of interest • e.g., to pursue Chen’s hypothesis that broad + rapid diffusion may signal really impactful papers • To characterize research knowledge diffusion patterns • Calculation issue • We use a threshold of at least 3 Citing SCs for a paper to get a Diffusion Score[could be 1 cite by a paper whose journal is associated with 3 SCs (rare)] • For Total score calculation this is less a factor than for Annual scores

14. Mean Annual Diffusion Scores for 6 Subject Categories For most of the 1995 benchmarks, Diffusion scores increase steadily with time. Mathematics is an outlier.

15. Mean NeuroScience Diffusion scores for all 4 benchmark years

16. Diffusion Patterns • Can we categorize research knowledge diffusion patterns? A first try here: • Steady growth • Staying power • Late-bloomer • Robust growth • Next 4 slides illustrate for those 1995 publications in the 6 SCs that fit these patterns

17. Steady Growth Typology Data Source: 1995 benchmarks

18. Staying Power Typology Data Source: 1995 benchmarks

19. Late-Bloomer Typology Powerhouse articles (N’s are small) tend to have high Diffusion scores Data Source: 1995 benchmarks

20. Robust Growth Typology Data Source: 1995 benchmarks

21. Diffusion Issues • Papers are citing more references over time -- does this exert upward pressure on Diffusion scores? • What happens to the speed of diffusion over time? - Will electronic open source publishing boost Diffusion? • How differently does research published in various Subject Categories diffuse? • Chen: papers which are cited quickly and diffusely are more likely to be blockbusters • Tool for tracking individual researchers or research groups’ influence

22. Taking a Look at the Research of Robert Nerem Institute Professor Emeritus and Director, Georgia Tech/Emory Center for Regenerative Medicine 240 Web of Science (WOS) publications, in 76 Journals, over 47 Years, and more than 8,000 Citations

23. Diffusion Scores v. Publication Age for Dr. Nerem’s Research Increasing Diffusion score from his early papers (high Age) to more recent ones

24. For more on Integration see: • www.idr.gatech.edu • Porter, Alan, Alex Cohen, David Roessner and Marty Perreault. "Measuring researcher interdisciplinarity." Scientometrics. 72.1 (2007): 117-147. • Porter, Alan, David Roessner, and Anne Heberger. "How interdisciplinary is a given body of research?" Research Evaluation. 17.4 (2008): 273-282. • Porter, Alan, and Ismael Rafols. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.

25. For more on Diffusion see: • Chen, Chaomei, and Diana Hicks. "Tracing knowledge diffusion." Scientometrics. 59.2 (2004): 199-211. • Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., & Pellegrino, D. (2009). Towards an explanatory and computational theory of scientific discovery. Journal of Informetrics, 3(3), 191-209 • Yu, G., Wang, M.Y., Yu, D.R. “Characterizing knowledge diffusion of Nanoscience & Nanotechnology by citation analysis.” Scientometrics, 84.1 (2010): 81-97. • Carley, S. and Porter, A.L. (2011) "A Forward Diversity Index." Scientometrics, forthcoming.

26. Thank you!

27. Mean Biotech Diffusion scores for all 4 benchmark years

28. Mean EE Diffusion scores for all 4 benchmark years

29. Mean Math Diffusion scores for all 4 benchmark years

30. Mean Med-R&E Diffusion scores for all 4 benchmark years

31. Mean Phy-AMC Diffusion scores for all 4 benchmark years