1 / 17

Juan D. Rogers School of Public Policy Georgia Institute of Technology

Network Effects In Citation Patterns: The Interdependence of Publications with High and Low Citation Rates. Juan D. Rogers School of Public Policy Georgia Institute of Technology. Presentation Outline. Background of Previous Citations Studies Normative Theory of Citations for Evaluation

butch
Download Presentation

Juan D. Rogers School of Public Policy Georgia Institute of Technology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Network Effects In Citation Patterns: The Interdependence of Publications with High and Low Citation Rates Juan D. Rogers School of Public Policy Georgia Institute of Technology

  2. Presentation Outline • Background of Previous Citations Studies • Normative Theory of Citations for Evaluation • Evidence for Interdependence of Highly and Lowly Cited Papers American Evaluation Association, Washington DC

  3. Nanotechnology Publications Data • Field defined by search strategy documented in: Porter, A., Youtie, J., Shapira, P., Schoeneck, D. 2008. “Refining Search Terms for Nanotechnology,” Journal of Nanoparticle Research, 10(5):715-728. • Well established definition of field boundaries • Paper analyzing first ten cohorts of emerging field • 1991 - 2000 • Yearly citations for each individual paper American Evaluation Association, Washington DC

  4. Features of the Distributions • Descriptives similar in spite of rapid growth of cohorts • Mean is significantly larger than the median • Median: in 10 years 50% papers have 10 citations • Mean: grows faster, 20 in 10 yrs 30 in 20 yrs. • Probability of above average citations: ~.25 • Very high probability of being cited at all • By our measure, “delayed” information is large in proportion to the total flow per cohort American Evaluation Association, Washington DC

  5. Delayed Citation for Top 2500 American Evaluation Association, Washington DC

  6. What is the value of citations? • Normative theory: • Highly cited papers are more valuable than papers with few citations • Suggestions to “ignore” most papers with few citations due to information “glut” • Often used to evaluate individuals • Few benchmarks to understand performance American Evaluation Association, Washington DC

  7. What if we “ignore” papers with few citations? • Conducted an experiment • Took one cohort of Nanotechnology • Year 2000 (N=26777) • Eliminated all papers with fewer than the median number of citations (10 citations in 2010; eliminated half the papers) • Then eliminated from remaining papers the citations from papers with fewer than the median for their cohort • Constructed a reduced-citations distribution of remaining papers American Evaluation Association, Washington DC

  8. Hypothesis: Distribution doesn’t change • If papers with few citations are ignored: • So should their citations • If they are worthless then so is the fact that they cite other papers • From previous results: • Power law distributions related to “fractal behavior” • New distribution should have same shape at new scale American Evaluation Association, Washington DC

  9. Some features of the 2000 cohort • Previous results (cumulative citations to 2010): • Mean: 26.34 • Median: 10 • Max: 2633 • Scaling factor: 2.87 • Distribution error: 0.014 • Tail minimum: 157 citations (top 530) American Evaluation Association, Washington DC

  10. Updated cohort 2000 (to 2013) • Features of the power law tail: • Scaling factor: 2.7 • Distribution estimation error: .0133 • Tail minimum: 170 (top 945) • Maximum citations: 4823 American Evaluation Association, Washington DC

  11. Cumulative Distribution Graph American Evaluation Association, Washington DC

  12. Features of “Reduced” Distribution • Scaling factor: 2.85 • Distribution estimation error: .0146 • Tail minimum: 104 (751) • Maximum citations: 1730 American Evaluation Association, Washington DC

  13. Cumulative Citation Graph (“Reduced” Citations) American Evaluation Association, Washington DC

  14. Comparing Both Graphs American Evaluation Association, Washington DC

  15. The ranks also change American Evaluation Association, Washington DC

  16. “Gateway” Phenomena American Evaluation Association, Washington DC

  17. Conclusions • Eliminating papers with few citations creates a new distribution of the same shape • Normative implication: • The process could then be repeated indefinitely • Ignores the network effect of citation • Papers with few citations play an important role: they signal field consensus • Network phenomena create “gateway” phenomena: • Papers with fewer citations are often cited by a few highly cited papers • Papers do not stand alone in value • Points again to the relatively slow flow of knowledge American Evaluation Association, Washington DC

More Related