Superiority of an error mastery approach further investigated: Findings from both the lab and the field

1. Superiority of an error mastery approach further investigated:Findings from both the lab and the field Cathy van Dyck The Cat’s Eye / VU University

3. Cafeteria Mind Your Head

4. Cafeteria Mind Your Head

5. “Employees have to put a stamp with their identification code on their work[…]. But they’re even smarter, they just don’t put down their identification code, so we don’t know who has made the mistake.” … “Checking, checking, checking. We are now thinking about installing video cameras.”

6. “[…] Learning from errors. I see them as positive, as free feedback. I always see them as positive. If people report their errors to me, I thank them for that. I don’t complain, I regard it as an opportunity to improve things. That how I see errors.”

8. Errors: What do we know? • Organizations can be differentiated on mastery and aversion aspects of error culture. • Eta2 = .33, p < .01 for Mastery • Eta2 = .38, p < .01 for Aversion • Mastery is related to company performance • positive correlation with both subjective (r = .33**) & objective (r = .46*) performance measures • relationship is upheld when controlling for age, size & line of industry • replication of results in Germany Van Dyck et al, JAP, 2005

9. Errors: What do we know? • More is learned from errors with severe consequences • Homsma et al. (in press), JBR • More constructive error handling in organizations that have a tolerant, yet decisive ‘basic view’ (assumptions) • Homsma (2007)

10. Overview • What are the underlying processes that explain superiority of mastery over aversion? • Experiments: • mediators of the error culture-performance relationship • What is the effect of setting (degree of risk)? • Field study: • Actual errors followed over time

11. Paradigm • Manipulation error approach • Error inducement: • Task paricipants are unfamiliar with • Too little instructions given • Practice task (relatively easy) • Performance task (difficult) • Questionnaire in between tasks • Task behaviour coded (video)

12. Experiments: manipulations • Mastery: “to err is human” A lot can be learned from errors. Errors point you at things you don’t know. So errors are really not bad. Try to learn as much as possible from your errors! • Aversion: “better safe than sorry” When working on a task, it’s important to have the right approach from the start. Errors have a tendency of escalation. If you strive to work without errors you can learn a lot!

13. Exp1: error approach & reinforcement • Manipulation error approach X reinforcement • reward/penalty points • N = 83 • Task: programming in BASIC • DV: task performance

14. Performance task: two main effects

15. Experiment 2: control & responsibility • Manipulation: error approach • N = 58 (dyads, 118 participants) • Task: programming in BASIC • All task behaviour video recorded & coded • DVs: • succesful correction of error (control) • blaming the partner (taking responsibility (R))

16. Error handling practice task

17. Error handling performance task

18. Percentage corrected

19. Taking responsibility: blaming the partner (rev.)

20. Experiment 3: attribution further explored • Manipulation: Locus X Stability • N = 80 • Task: Lemmings (computer game) • Experienced control over and responsibility for error causes. • DV: Quality strategy Homsma et al. (2007), JB&P

21. Perceived control cause: interaction

22. Quality strategy: interaction

23. Experiment 4: self-regulation • Manipulation: error approach (+ control group) • N= 68 • Task: Tower of London, with error inducement to minimize variation in number of errors, and avoid learning benefits from mastery approach. • DV: Self-focussed attention (Bagozzi & Verbeke, 2003).

24. Self focussed attention

25. Experiment 5: self-focussed attention • Manipulation: SFA vs noSFA (+ control group) • Be filmed (SFA) • Filler info (no SFA) • Write short piece (control) • N= 68 • Task: Boloball (computer game) • DVs: Strategy use Performance improvements

26. Competitive strategy (blocking opponent)

27. Performance increase

28. Results: Summary • Pure effect error approach (Exp.1) • Less control of consequences & taking responsibility under Aversion instructions (Exp. 2) • Better strategies with controllable causal attribution (Exp. 3) • Error approach affects SFA (Exp. 4) • SFA negatively affects strategy and performance gains (Exp. 5)

29. Does error management work every where? “[…] I know the mountain [Everest] is an environment so extreme there is no room for mistakes.” (Cahill, 1997, p.245-246). • The influence of the environment

30. Errors & environment • Severe errors? .. do not exist. • Krakauer (Adventure Consultants exp. 1996) • The environment affects the likelihood of severe consequences (risk).

31. Field study: Error tracking over time • Testing the effects of error approach • in real error incidents (T1) • with T2 follow-up • … on control • reduction of negative error consequences • and learning • single-loop: improvements • double-loop: integration new insights & ideas

32. Error Tracking Instrument (ETI) • Description of a recent error • Error handling • communication (T1) • clarity cause (T1) • Control • (possible) consequences (T1) • reoccurrence (T2) • Learning • trouble shooting/LT solution (T1) • improvements (T2) • new insights (T2)

33. Error tracking: Research settings • Innovative e-business company • Low risk • Nurses of emergency unit • High risk

34. Examples of errors • e-business: • Overlooking a big lay-out error in an expensive advertisement. • The sales department selling a service for which there is no knowledge of implementation nor capacity to support it. • Erroneous billing of hours to a client. • Installing an excellent, but for this client, totally useless application.

35. Examples of errors (2) • EU nurses: • A patient falling of a hospital trolley. • A blood sample mix up. • Using an expired drug.

36. Error incidents (1) • Wrong assessment of situation. Illness of patient in the emergency room deteriorated. The patient got into a shock. • Error was made upon arrival of patient. The error was discovered and corrected three hours later. The patient would otherwise have died. • The supervisor was not informed, but a colleague was. Colleagues reacted uninterested.

37. Error incidents (1 cont’d) • The error was not further discussed within the department, because people feared for their position. This was especially salient as there was rumor of a merger. • At the measurement three weeks later, the respondent reported that no measures had been undertaken since, nor had new insights emerged. The error had not reoccurred in the weeks after its first occurrence.

38. Error incidents (2) • Taking up and starting work on a new project for an important client. There was a verbal agreement, but no contract had been signed by the client yet. Due to financial difficulties, top management of the client company was unwilling to sign, or to pay for the work that had already been done. There was a strong (legal) case to make the client pay, but playing "hardball" might be undesirable as it would probably result in losing this client all together.

39. Error incidents (2 cont’d) • The problem was recognized two weeks into the project, but no action was taken for four more weeks. • For the IT company, this problem meant a loss of income, for the employees involved it meant not attaining a bonus. • Supervisors were informed and responded emphatic. Without laying blame, the problem was further discussed with all people involved, which resulted in a decision to cut the losses, while making sure that this problem could not reoccur.

40. Error incidents (2 cont’d) • One of the reasons that this could have happened, they concluded, was a flawed communication process, lack of cooperation and consulting each other. • Contract managers, who served as a central contact for clients were appointed and check ups were implemented for contract statuses. Further, the sales process was improved. It turned out to be an important lesson for every one.

41. ETI: Error Handling • Communication • 1 (e.g., “I have not discussed this error with anyone.”) through 5 (e.g., “We thoroughly discussed it in order to handle matters in the best possible way.”; kappa = .91). • Clarity cause • 1 (e.g., “We did not analyze it. It hasn’t become clear what happened.”) through 3 (e.g., “Yes, boundaries are clear now.”; kappa = .89). • Correction • 1 (e.g., “The error was not corrected, maybe it will go better next time.”) through 3 (e.g., “Yes, we corrected through alternations in the system.”; kappa = .97).

42. ETI: Control • Severity of error consequences (T1) • 1 (e.g., “There were hardly any consequences”; “The patient just had to wait a bit longer.”) through 5 (e.g., “Serious dispute with client”; “Patient got into shock”; kappa = .93). • Containment of consequences (T1) • 1 (e.g., “No, the consequences could not have been more serious”) through 3 (e.g., “Consequences could certainly have been much more serious, if the error had been detected later”; kappa = .82). • Error reoccurrence at T2 was measured by a dichotomous (no/yes) scale.

43. Learning from error • Long-term solution (T1) ranged from 1 (trouble shooting only) to 3 (long term solution and improvement of processes; kappa = 1.00). • Improvements at T2 ranged from 1 to 5 • New ideas at T2 ranged from 1 to 5

44. Error tracking: Control & learning

45. Results: T1 => T2 • Error handling at T1 predicts learning at T2: • Improvents predicted by Communication (r = .35*) and Clarity cause (r = .29#), • Integration new insights predicted by Communication (r = .36*) and Clarity cause (r = .27#)

46. “When I first started as a supervisor, I used to get angry at people when they made a mistake. That’s very easy and seems forceful. But you have to learn that it simply doesn’t work. People will get frustrated, fearful, they will be less open about their mistakes and therefore errors will be discovered later.” … “I try to create an open atmosphere and tell people that they should tell me when they have made a mistake so that we can do something about it. We try to be open and discuss errors, because we believe that is the only way we can control damage.”

47. More info? • www.thecatseye.nl • vandyck@thecatseye.nl

48. Error - goal not attained - not atributable to chance - could have been avoided in short: - "good" intentions - failed action versus versus Violation - conscious decision to disobey rules in short: - "wrong" intentions - not necessarily failed action Consequences either - negative (e.g. time loss, costs, accident etc) or - positive (e.g. new information, innovation, future error prevention)