1 / 22

Ilya Ponomarev 1 , Pawel Sulima 1 , Jodi Basner 1 , Unni Jensen 1 , Joshua Schnell 1 ,

A New Approach for Automated Author Discipline Categorization and Evaluation of Cross-Disciplinary Collaborations for Grant programs. Ilya Ponomarev 1 , Pawel Sulima 1 , Jodi Basner 1 , Unni Jensen 1 , Joshua Schnell 1 , Karen Jo 2 , and Nicole Moore 2. ilya.ponomarev@thomsonreuters.com.

jenaya
Download Presentation

Ilya Ponomarev 1 , Pawel Sulima 1 , Jodi Basner 1 , Unni Jensen 1 , Joshua Schnell 1 ,

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A New Approach for Automated Author Discipline Categorization and Evaluation of Cross-Disciplinary Collaborations for Grant programs Ilya Ponomarev1, Pawel Sulima1, Jodi Basner1, Unni Jensen1, Joshua Schnell1, Karen Jo2, and Nicole Moore2 ilya.ponomarev@thomsonreuters.com 1Custom Analytics, Rockville, MD 2National Cancer Institute, Bethesda, MD 10/16/2013 5:30 PM

  2. Why Cross-disciplinary Research? “Interdisciplinary research can be one of the most productive and inspiring of human pursuits” Facilitating Interdisciplinary Research National Academy of Sciences, 2005 • Innovation increasingly occurs at the boundaries of disciplines • Complex “Puzzles” require diverse background • Data avalanche from multiple sources requires fusion of information • Convergent technologies require integration across disciplines

  3. US Government Funding of Cross-disciplinary R&D DOD DOE NSF NIH NASA

  4. How to Measure Success of Cross-disciplinary Program? See also J. Basner, “Evaluating Collaboration and Outcomes of Health Research” Friday, 10/18/2013, 11:00am at Gunston East Rm THIS TALK: • In order to measure cross-disciplinarity define disciplines as accurate as possible • General approach of automatic assigning grant specific categories to papers and people • Application to NCI PS-OC grant program classification?

  5. Institute Facilitate Generate NCI Physical Sciences-Oncology Centers 09/2009-Current 12 centers, 250 Researchers

  6. Evaluation: Birds View productivity Fields convergence collaboration impact J. Basner, Friday, 10/18/2013 • Compare baseline data set (2006-2008) with ongoing research data set (2009-2012)Web of Science+ Medline 2009-2012: 601 reported pubs 2006-2008: 3,367 pubs 166 active PS-OC investigators 202,000 references 4,199 journal titles • Use publications as a proxy of outcome

  7. Evaluation: Birds View PS-OC 2/3 broad categories Oncology Life Sciences Physical Sciences Approach:

  8. 266 Web of Science Journal Subject Categories PS-OC 3 broad categories Oncology Life Sciences Physical Sciences • Has Oncology SC • Multiple SCs per journals (up to 7) • Multidisciplinary (meaningless, but “Science”, “Nature”) • Some SCs are already inter-disciplinary • LSs dominates after aggregation

  9. 22 ESI Subject Categories • One SC per journal • Does not have Oncology • Multidisciplinary SC exists also • Clinical medicine? • LSs dominates after aggregation

  10. Mapping. Challenges PS-OC 3 broad categories Web of Science 266 Journal SCs Web of Science 22 Broad ESI categories • One SC per journal • Does not have Oncology • Multidisciplinary SC exists also • Clinical medicine? • LSs dominates after aggregation • Has Oncology SC • Multiple SCs per journals • Multidisciplinary • Some SCs are inter-disciplinary • LSs dominates after aggregation Oncology Life Sciences Physical Sciences Approach: • Intermediate map on extended 6 Broad Categories • Paper level SC assignment based on references

  11. Step 1. Introduce 6 Intermediate PS-OC Categories for Better Selection: (very often MED journals are closer to ON than LS) Will be dropped on final stage PS – Physical SciencesLS – Life SciencesOC – OncologyMED – Medicine OTH – Others MULT – Multidisciplinary 11

  12. Step 2. Map 265 WoS JSC to 6 PS-OC Categories: (usually published in “Nature”, “Science” or “PNAS”) Meaningless in terms of assignment PS-OC category: article published in MULT journal can be about PS, or about LS, or OC. Usually, it is not interdisciplinary article. Additional re-classification of article’s research field is needed based on references. Examples: a) Obvious: Acoustics  PS, Chemistry, Analytical  PSOncology  OC, Management OTH b) Dominant:Biophysics  PS c) Dominant:Physics, Multidisciplinary  PS d) Meaningless:Multidisciplinary  MULT

  13. Step 3. Assign PS-OC Categories Weights to Each Journal Each journal should be counted equally LS=1/2PS=1/2OC=0MED =0MUL=0OTH=0 Biology LSBiophysics  PSRadiology, NM PS LSPS 2 Select distinctPS-OC categories Map Count total (denominator) Weights) (Journals in WoS can have 1 or 2, or 3, … even 7 SCs) • Examples:Journal “Radiation Research” – 3 SCs:

  14. Step 4. Calculate combined J-R weights for publications: Better assignment of paper’s field based oninformation what paper cites Journal weights Aver. Refs Weights ½ (Journal + Refs) LS=0PS=0MED=1OC=0MUL=0OTH=0 LS=0.23PS=0.04MED=0.17OC=0.36MUL=0.19OTH=0 LS=0.12PS=0.019MED=0.58OC=0.18MUL=0.1OTH=0 Example:Coffey D., Getzenberg R. JAMA, 2006 1 journal cat (MED=1) 26 Refs: 14

  15. Step 5. Collect all publications for each investigator, calculate average weights, and rank PS-OC categories: Person Inter-disciplinarity Averaged J-R Weights Ranks LS =0.32PS =0.04MED=0.23OC =0.41OTH =0.01 LS =2PS =4MED=3OC =1OTH =5 3 Example. DavidA  8 pubs: Average JR weights

  16. Step 6. Redistribute MED and OTH weights between OC,LS, and PS LS =0.32PS =0.04MED=0.23OC =0.41OTH =0.01 LS =0.4PS =0.05OC =0.55

  17. Validation At the beginning of the program: Investigators self-nominated themselves as oncologists or physicists

  18. Applications: how publication patterns change

  19. Future Development PhysicalScientist Oncologist Life Scientist PS-OC Network Investigators Outside Network Co-authors 19

  20. Conclusions • Automated approach for decomposition of scientific publications into grant specific discipline categories • Multi-step method with intermediate mapping • Weighted SC assignment based on article’s and its references’ SCs • Precision-recall validation based on investigators’ self-categorizations • Oncologists within the NCI’s PS-OC program are publishing more physical sciences research and physical scientists are publishing more oncology or life sciences research during years of program participation.

  21. ilya.ponomarev@thomsonreuters.comThomson ReutersCustom AnalyticsRockville, MD

  22. SUPPORTING SLIDES

More Related