Management-Based Evidence

1. Management-Based Evidence SIOP New Orleans April 2, 2009 Benjamin Schneider, Ph.D. Senior Research Fellow, VALTERA

2. Outline • Putting the burden on researchers. This talk is not for you if you could care less about management paying attention to your findings. • Issues to consider • Sponsor characteristics • Strategic Focus • Doability • Administration and data analysis • Receptivity • Feedback • Follow-through and follow-up • Summary and Conclusion

3. Putting the Burden on Researchers • Evidence-based management puts the burden on management • Don’t operate on hunches • Do operate on evidence • Misses the paradox: • Management achieves by having good hunches (Gladwell) • Researchers know what should be done; management knows whether it can be done • Management knows the nuances for local implementation • Conclusion: Listen carefully to your sponsor before beginning the research if you want the data to be used

4. Sponsors • Permit you to do research and/or hire you because they like you; your credentials are secondary • Sponsors are the key to action after the research; you must please them through the process or they will not use your findings • Sponsors must be intimately consulted in every stage of the process so they feel comfortable with the resultant findings: • They must understand the findings in their terms so you must know who they are and what there terms are • They lay their career on the line to implement findings so listen very carefully

5. Strategic Focus • If you do not show how your work is directly, or at worst indirectly related to your sponsor’s critical problem fugedaboudet. • Sponsor wants an employee opinion survey—find out why. • Sponsor wants a new selection process; find out what the present one is not doing • sponsor wants you to help her make change—find out from what, to what, why that, and why now? • Generic anything we do is out; strategic everything we do is in. A strategic focus is how management thinks and so should we: • Employee opinion surveys • Personnel selection • Goal-setting and rewards systems • Climate and culture efforts at measurement and change

6. Strategic Focus Examples • Organizational climate-climate for what? • Service • Safety • Innovation • Employee engagement towards what ends? • Customers? • Employee Turnover • Personnel selection to achieve what goals? • Efficiency • Quality

7. Doability • Financial resources? • Lexus vs. Fit • Customized? • CEO commitment—how far and in what ways? • Public sign-off on change? • Make the project as important as financial results? • Need for a line-based project team, members of which are like those who will be affected • Communications to anyone/everyone who will be involved • Touching base to encourage line support throughout • Tracking project process, data analysis and feedback preparation

8. Receptivity • Project goals in the language and conceptual space of the organization • Careful diagnosis of the key strategic goals, norms, roles & functions • Mission statement, myths and stories and their focus and message • Introduce all projects so they “fit” the context—good use of the project team • Implemented tools must also fit the context • Deal head on with cynicism associated with previous efforts • Surveys in the language of the context • Tests, interviews and simulations that are highly relevant to the context • Face validity saves face • Line interviewers and assessors • Have no secrets—because there are no secrets

9. Administration and Data Analysis • Administration • Preceded by letter from CEO about the importance attached to the project—and that there will be follow-up (unlike previous surveys) • If paper, administer in groups • If on-line get time off to do it • Data analysis • Management likes item data—”top box” or “top boxes” • Compare item data to the competition (norms) • testing theory using scales is fine but not relevant to management • Driver analysis • Do driver analysis against strategic outcomes • Do driver analysis at the level of the team or group or unit where action happens • Focus in on actionable items (NOT: Overall how satisfied …) • Driver analysis focuses on drivers, not just low-scoring items

10. Feedback • Invite the CEO and his or her TMT so they hear the presentation and witness the reactions of the unit leaders • Test the findings and implications with the project team first • Always present the good news first and that the purpose of everything is to build on strengths • Put the most important findings up front, with implications • Present the process by which the evidence was gathered • Arrange a separate meeting with the CEO to discuss change issues • Unit leaders give feedback to their units • Train them to do the feedback with the focus on the positives and the future and not the past • Make unit leaders responsible for presenting their action plans to at least the project team and hopefully the CEO (or his or her representatives)

11. Summary and Conclusions • Evidence-based management puts the burden on management to take action based only on evidence • Researchers should shoulder the burden of producing management based evidence • Management-based evidence is: • generated in the world of managers • focuses on their strategic needs • pays far more attention to the research process than is typical • presents data in ways that managers find useful and understand • operates with a basic philosophy of change as the outcome • All of my data based publications have been paid for by management so this approach is not a limitation on publishing