KT (CIHR definition) Terms Presentation

1. How we use the language of KT and its challenges: Time for changes?Ann McKibbonMcMaster University

2. KT (CIHR definition) Terms Presentation • Set projects in context • How many KT terms? • How we use KT terms in papers? • Can we use KT terms for information retrieval? • Does anyone care about our KT terms? • KT Term wiki • Next steps in term consolidation?

3. Funding National Coordinating Centre for Methods and Tools (PHAC) CIHR KT Synthesis Grant KT Canada People Donna Ciliska Dave Davis Maureen Dobbins Brian Haynes Cynthia Lokker Sharon Straus Nancy Wilczynski (Ian Graham) Enablers

4. Projects • Collect KT related terms and their definitions • Use these terms to construct searching filters for KT material • Use the terms and definitions to analyze them • Possibly make recommendations for next steps We approached these projects from an information science perspective.

5. Collecting KT Terms and Definitions Building on previous work (eg Graham, Estabrooks et al) Searching • Databases • Textbooks • Google • Google scholar • Existing articles • Discussion with KT people • Citations in articles

6. Collecting KT Terms and Definitions 100 terms related to KT for projects 109 terms now 72 strongly related to KT 37 with weaker relationship with KT Could be many more • Word order, spelling, endings, etc • Audit and feedback 187 instances in Medline • Feedback and audit 825 instances • Utilization 195K instances • Utilisation 6K instances

7. Conclusion 1 We use a lot of terms related to KT They differ according to: …time period …geography …discipline …what is being studied (individuals, decision makers, policy experts)

8. Search Filters

9. Searching Filter Examples 52 vs 168

10. Can we make filters for KT material? To set up filter studies: Hand search (selected journals) All citations tagged Yes or No (eg KT or RCT) List of terms and phrases for testing These words are individually tested: does it retrieve the “Yes” articles? does it NOT retrieve the “No” articles? Combine terms and test their retrievals Sort out which strategy is best for retrieving the most “Yes” articles (sensitivity) retrieving the fewest “No” articles (specificity) the best balance between sensitivity and specificity

11. 12 Handsearch Journals Journal # articles #KT articles %KT BMJ 520 150 29% Ann Intern Med 268 116 43% JAMA 312 87 29% Soc Sci & Med 530 87 16% J Adv Nursing 265 47 18% Health Affairs 205 40 20% J Med Libr Assoc 59 16 27% Addiction 168 13 8% Int J Nurs Practice 48 9 19% J Occupat Env Med 148 8 5% Nursing Research 49 7 14% Nursing Inquiry 31 1 3%

12. Subdivided KT (“Yes” articles) All articles that had KT content broadly defined These KT articles further divided: KT application articles (real project with data) KT theory articles (model, theory, framework) Application KT Theory All KT

13. Medline Results (CINAHL Separate) 12 journal titles for 2006: 2596 articles (no letters, editorials) 581 articles on KT 201 KT application (real projects) 153 KT theory Duplicate reading with resolution by consensus Articles could be in multiple categories

14. Terms for Searching KT • Index terms assigned by indexers • Words and phrases used by authors • 100 terms related to KT • Lots of other terms that might also be related to KT or KT methods • Multiple versions of these • Endings (plurals, -ing, -ed) • Spellings (UK, US) • Term order including different adjacencies • Single words in phrases and phrases • Tested 6502 single terms/term phrases • Alone (sensitivity, specificity, accuracy) • Combinations • Double • Triple • Quadruple

15. Goal for retrieval Should be > 90% for sensitivity and specificity Filter for systematic review articles in Medline meta analysis.mp,pt. or review.pt. or search:.tw. Sensitivity 98% miss 2% of needed Specificity 91% some but not a lot of irrelevant material

16. Analysis and Results (creative commons)

17. Medline Single Term Retrieval For all KT material sensitivity specificity Patient:.mp. 53.2% 64.2% For KT applications Use:.tw. 55.7% 62.9% For KT theory exp health services administration/ 56.6% 59.8%

18. Medline 4-term Strategies For all KT material exp clinical medicine/ or exp health services/ or exp review literature/ or patient.mp. Sensitivity 82.7% miss 17% of needed Specificity 53.1% huge number of irrelevant articles

19. Medline 4-term Strategies For all KT material 82.7% 53.1% For KT applications evidence-based medicine.mp. or exp delivery of health care/ or exp health services/ or randomized controlled trial:.mp. Sensitivity 85.1% miss 15% of needed Specificity 50.4% huge number of irrelevant

20. Medline 4-term Strategies For all KT material 82.7% 53.1% For KT applications 85.1% 50.4% For KT theory exp delivery of health care/ exp evaluation research/ or medicine.mp. or research.tw. Sensitivity 90.8% miss 9% of needed Specificity 52.0% huge number of irrelevant

21. Conclusion 2 We cannot not use our available KT terms or indexing words to easily retrieve… …KT articles …articles describing KT applications …only a tiny bit better for KT theory

22. Hint: Seed your abstracts …This evidence-based medicine (ebm) study used concepts from knowledge translation (aka research utilization, quality improvement, translational research, and continuing professional development (continuing education)) for evidence-informed decision making and dissemination…

23. How do authors use KT terms? 581 KT articles (medicine, nursing, LIS, business) 2015 nonKT articles 100 KT related terms In the abstracts of these 581 and 2015 articles • How many of the 100 terms were used? • Used differentially in KT and nonKT articles?

24. 44 terms used 56 terms not used 17 used 10 or more times 7 terms used 5-9 times 2 terms used 4 times 4 terms used 3 times 7 terms used 2 times 14 terms used 1 time Top 10 terms Use: 1000 Change: 359 Information 317 Change 197 Policy 165 Evaluation 131 Communication 84 Policies 67 Implementation 63 Utili(z/s)ation 52 Adoption 31 100 Terms and 2596 articles

25. Did the 44 KT terms differentiate between KT and nonKT articles? All KT articles 8 KT terms had high discrimination power 7 KT terms had medium discrimination power 3 KT terms had low discrimination power 26 KT terms had no discrimination value (don’t forget 56 terms not used) KT application articles 13 high 8 medium 4 low 19 no discrimination KT theory articles 18 high 6 medium 3 low 17 no discrimination High Medium Low No P < 0.001 P < 0.01 to 0.001 P < 0.05 to 0.01 P = or > 0.05

26. Conclusion 3 We do not use our available KT terms … ….very often in our articles ….consistently in our KT articles ….even those we use are not used often ….the ones we use not often unique to KT

27. Next Step: KT of KT terms We now have 109 KT related terms Are there any more? We also have their definitions? How can we make this information available?

28. Wiki to present terms/definitions Wiki

29. WhatisKT wiki http://whatiskt.wikispaces.com/

30. Knowledge Transfer (23 definitions)

31. KT Term Wiki: Visitors

32. KT Term Wiki: Countries

33. KT Term Wiki: Countries Sept/08

34. KT Term Wiki: Countries Sept/09

35. Most Viewed Terms (1-10)

36. Most Viewed Terms (11-20)

37. Least Viewed Terms

38. Wiki: Messages/Suggestions

39. Conclusion 4 • Substantial interest is occurring in looking at KT terms and their definitions • Substantial interest from US audiences • Not a lot of interest in helping develop the terms

40. Next Steps? Need to work on our terms with respect to • Understand them • Use them consistently • Consolidation? • Improve retrieval • Seek help from large database producers to improve indexing and indexing terms • Getting help from other disciplines and domains • Linguistics • Philosophy

42. WHO definition of Drowning ABSTRACT Experts in clinical medicine, injury epidemiology, prevention and rescue from all over the world participated in a series of "electronic" discussions and face-to-face workshops. The suitability of previous definitions and the major requirements of a new definition were intensely debated. The consensus was that the new definition should include both cases of fatal and nonfatal drowning. After considerable dialogue and debate, the following definition was adopted: "Drowning is the process of experiencing respiratory impairment from submersion/immersion in liquid." Drowning outcomes should be classified as: death, morbidity, and no morbidity. There was also consensus that the terms wet, dry, active, passive, silent, and secondary drowning should no longer be used. Thus a simple, comprehensive, and internationally accepted definition of drowning has been developed. Its use should support future activities in drowning surveillance worldwide, and lead to more reliable and comprehensive epidemiological information on this global, and frequently preventable, public health problem. van Beeck EF, Branche CM, Szpilman D, et al. A new definition of drowning: towards documentation and prevention of a global public health problem. Bull World Health Organ. 2005 Nov;83(11):853-6. PMID: 16302042; PMCID: PMC2626470.

43. Mitroff II, Sagasti F. Epistemology as general systems theory: An approach to the design of complex decision-making experiments. Philosophy of the Social Sciences 1973, 3:117-34. "In many respects, the most troublesome problems of any science centre around its most basic terms and fundamental concepts, and not around its more sophisticated concerns. Indeed to the extent that everything either follows from or is based on a discipline's basic terms and fundamental concepts, problems at a higher level can always be traced back to problems at a more fundamental level."