Evidence-Based Management: An Introduction

1. Evidence-Based Management An introduction Hungarian Association of Executives October 26, Budapest

2. Where does it come from? • What is it? • Why do we need it? • How does it look like in practice? • (An example)

3. Evidence based management: Where does it come from?

4. What field is this? • “there is a large research-user gap” • “practitioners do not read academic journals” • “the findings of research into what is an effective intervention are not being translated into actual practice” • “academics not practitioners are driving the research agenda” • “the relevance, quality and applicability of research is questionable” • “practice is being driven more by fads and fashions than research” • “many practices are doing more harm than good”

5. Medicine: Founding fathers David Sackett Gordon Guyatt McMaster University Medical School, Canada

6. How it all started

7. Problem I: too much information • More than 1 million articles in 40,000 medical journals per year (= 1995; now probably more than 2 million). For a specialist to keep up this means reading 25 articles every day (for a GP more than 100!) • Most of the new insights and treatment methods don’t reach the target group

8. Problem I: too much information • HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a ‘general’ manager more than 50!)

9. Problem II: persistent convictions if you’re hyperventilating breathe into a bag

10. Problem II: persistent convictions elderly people who have an irregular heartbeat are much more likely to die of coronary disease give them a drug that reduces the number of irregular beats

11. How 40,000 cardiologists can be wrong In the early1980s newly introduced anti-arrhythmic drugs were found to behighly successful at suppressing arrhythmias. Notuntil a RCT was performed was it realized that, althoughthese drugs suppressed arrhythmias, they actually increasedmortality. The CAST trial revealedExcess mortality of 56/1000. By the time the results of this trial were published, at least100,000 such patients had been taking these drugs.

12. David Sackett • Half of what you learn in medical school will be shown to be either dead wrong or out-of-date within 5 years of your graduation; the trouble is that nobody can tell you which half. • The most important thing to learn is how to learn on your own: search for the evidence! • (Remember that your teachers are as full of bullshit as your parents)

13. Evidence-Based Practice 1991Medicine 1998Education 1999Social care 2000Nursing 2000Criminal justice ????Management?

14. Evidence based management: What is it?

15. Definition Evidence-based management means making decisions about the management of employees, teams or organizations through the conscientious, explicit and judicious use of four sources of information: 1. The best available scientific evidence 2. Organizational facts, metrics and characteristics 3. Stakeholders’ values and concerns 4. Decision supports and practitioner expertise

16. Four sources

17. Definition When making an important decision, an evidence-based manager knows whether there is scientific evidence available to support this decision (and how ‘strong’ the evidence is).

18. Evidence-based management: Why do we need it?

19. Four sources

20. Trust me: 20 years of management experience!

21. Errors and Biases of Human Judgment

22. Errors and Biases of Human Judgment • Seeing order in randomness • Mental corner cutting • Misinterpretation of incomplete data • Halo effect • False consensus effect • Attribution error • Group think • Self serving bias • Sunk cost fallacy • Cognitive dissonance reduction • Confirmation bias • Authority bias • Small numbers fallacy • In-group bias • Recall bias • Anchoring bias • Inaccurate covariation detection • Distortions due to plausibility

23. “I’ve been studying intuition for 45 years, and I’m no better than when I started. I make extreme predictions. I’m over-confident. I fall for every one of the biases.”

24. Errors and Biases of Human Judgment • Doctorsand managers hold many erroneous beliefs, not because they are ignorant or stupid, but because they seem to be the most sensible conclusion consistent with their own professional experience!

25. Errors and Biases of Human Judgment stress & lifestyle peptic ulcer

26. Oct 2005 Peptic ulcer – an infectious disease! This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By using technologies generally available (fibre endoscopy, silver staining of histological sections and culture techniques for microaerophilic bacteria), they made an irrefutable case that the bacterium Helicobacter pylori is causing disease. By culturing the bacteria they made them amenable to scientific study. In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is now firmly established that Helicobacter pylori causes more then 90% of duodenal ulcers. The link between Helicobacter pylori infection and peptic ulcer disease has been established through studies of human volunteers, antibiotic treatment studies and epidemiological studies.

27. Beliefs vs Evidence “What gets us into trouble is not what we don't know, it's what we know for sure that just isn't so.” Mark Twain

28. True or false? Incompetent people benefit more from feedback than highly competent people. Task conflict improves work group performance while relational conflict harms it. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals.

29. How evidence-based are we? HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008) • 959 (US) + 626 (Dutch) HR professionals • 35 statements, based on an extensive body of evidence • true / false / uncertain • On average: 35% - 57% correct

30. FADS

31. Evidence-based? • Competence management • Excellent care • Total Quality Management • Flexible workspace / The New World of Work • Knowledge Management • Investors in People, Great Place To Work • Balanced Score Card / INK • Lean / Six Sigma / TOC

32. Evidence-based approach How ‘new’ is this question or problem? Is there evidence from scientific research that can help us find the most effective approach? How ‘strong’ is this evidence? Is the evidence applicable to my situation?

33. Evidence based management: How does it look like in practice?

34. The 5 steps of EBP • Formulate a focused question (Ask) • Search for the best available evidence (Acquire) • Critically appraise the evidence (Appraise) • Integrate the evidence with your managerial expertise and organisational concerns and apply (Apply) • Monitor the outcome (Assess)

35. 1. Formulate a focused question

36. Asking the right question? • Does team-building work? • Does the introduction of self-steering teams work? • Does management development improve the performance of managers? • Does employee participation prevent resistance to change? • Is 360 degree feedback effective?

37. Focused question? • What is a ‘team’? • What kind of team? • In what contexts/ settings? • What counts as ‘team-building’? • What does ‘work’ mean? • What outcomes are relevant? • Over what time periods? • Does team-building work?

38. Answerable question: PICOC P= Population I = Intervention or success factor C= Comparison O= Outcome C = Context

39. Focused question: PICOC P= Population I = Intervention or successfactor C= Comparison O= Outcome C = Context • Employee productivity? • Job satisfaction? • Return on investment? • Market share? • Organizational commitment?

40. 2. Finding the best available evidence

41. Where do we search?

42. Hands on instruction

43. 3. Critical appraisal of studies Making sense of evidence

44. Best available evidence?

45. Research designs What is the BEST car?

46. Research designs Which design for which question?

47. Which design for which question? Explanation

48. Best research design?

49. Best available?

50. What is evidence? • Evidence is not the same as ‘proof’ or ‘hard facts’ • Evidence can be • - so strong that no one doubts its correctness, or • - so weak that it is hardly convincing at all