1 / 50

Chapter 5

Chapter 5. Human Error? No, Bad Design. Today’s Topics. Humans, systems, and error More on data gathering Designing for communication and coordination. Part 1: Humans, Systems and Error. Root cause analysis for error Why did Air Force stop at human error?

Download Presentation

Chapter 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Chapter 5 Human Error? No, Bad Design

  2. Today’s Topics • Humans, systems, and error • More on data gathering • Designing for communication and coordination

  3. Part 1: Humans, Systems and Error • Root cause analysis for error • Why did Air Force stop at human error? • Do not view the system being designed as one combining human and computer/machine actors (e.g. sociotechnical or cyber-human system) • The five whys method • Humans are good at creative and adaptive behavior. The more mechanistic the activity is required to be, … • Deliberate Violations • Occur in all types of contexts (e.g. driving, officework, security, …) • Sometimes because official procedures are developed for legal or accountability reasons • In many situations, work cannot get done without exceptions

  4. Errors: Slips and Mistakes • Slips: the goal is correct but the actions not done correctly • Action based – inaccurate performance of planned action • Memory lapse – forgetting to take a planned action • Mistakes: the goal or plan is wrong • Rule based – wrong plan due to poor planning (“wrong rule”) • Knowledge based – wrong plan due to inaccurate/incomplete knowledge • Memory lapse – forgetting at stage of goals, plans, or evaluation • Examples from elevator • Pressing button below correct one along with correct one • Pressing the 1st floor button when going to the lobby

  5. Slips • Slips are more common for experts than novices • Result of more aspects of a task being handled unconsciously • Capture slips • When similar action sequences get confused • Generally a more familiar or recent action sequence replaces another • Example might be finishing a password with more common suffix • More common for experts • Description-similarity slips • Performing an action with a similar item to the one intended • Designers should reduce similar items and reduce number to choose from • Mode errors • Occurs when design has modes where the same user action results in different system actions • Designers should try to avoid modes and make the mode visible when they are necessary • Memory lapse slips • Forgetting where one is in procedure (repeating or not performing steps) • Often caused by interruptions • Designers can show steps and indicate where one is or can create forcing functions enforcing prerequeisits

  6. Mistakes • Rule-based mistakes • Situation is misinterpreted so wrong rule is selected • Correct rule is invoked, but it is faulty or did not account for current situation • Correct rule is invoked, but the outcome is incorrectly evaluated • Knowledge-based mistakes • Occurs in situations requiring more general problem solving • Can be due to a mistaken mental model • Designers need to provide clear conceptual models • Systems need to support users when they encounter previously unknown situations • Cooperative problem solving system • Memory-lapse mistakes • Occur when user forgets previously gathered information about situation, or forgets the plan altogether

  7. Social Issues and Error • Social, time, and economic pressure lead to bad decisions • Social pressure also can lead to errors not being reported • Adding more responsible people can cause less attention to tasks • Each may assume others are being more responsible • Need to encourage reporting of errors to fix their causes • Jidoka and Poka-Yoke • NASA aviation safety reporting system

  8. Detecting Errors is Non-Trivial • Errors are generally preceded by a number of uncommon events that each by itself is not strongly indicative of error • False alarm • Response to correct action interpreted as failure • May occur • When there are conflicting indicators • Due to different interpretations of plan/task • Garden Path • Incorrect action not discovered until later • Losing context of original problem • Machine can interpret action as correct for some alternative path • Users can assume they know the process without machine • Trivial breaches of understanding can become “fatal”

  9. Designing for Error • Warnings • Need to be clear from one another and coordinated with one another • Need to gain needed attention without being obnoxious/disabilitating • Computer systems can often add • Different visual cues and locations for different actions/features • Sensibility checks on user actions • Undo and multi-stage undo • Checklists • Common in aviation but not other industries • Design of checklists is still a difficult task • More generally, processes can focus on creating resilience • The Swiss cheese model to minimize likelihood • Notifications when features are lining up for error

  10. Part 2: More on Data Gathering • Interviews • Questionnaires • Observation • Choosing and combining techniques

  11. Four key issues • Setting goals • Decide how to analyze data once collected • Relationship with participants • Clear and professional • Informed consent when appropriate • Triangulation • Use more than one approach • Pilot studies • Small trial of main study

  12. Data recording • Notes, audio, video, photographs • Notes plus photographs • Audio plus photographs • Video

  13. Interviews • Unstructured - are not directed by a script. Rich but not replicable. • Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness. • Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability.

  14. Interview questions • Two types: • ‘closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’ • ‘open questions’ do not have a predetermined format • Closed questions are easier to analyze • Avoid: • Long questions • Compound sentences - split them into two • Jargon and language that the interviewee may not understand • Leading questions that make assumptions e.g., why do you like …? • Unconscious biases e.g., gender stereotypes

  15. Running the interview • Introduction– introduce yourself, explain the goals of the interview, reassure about the ethical issues, ask to record, present any informed consent form. • Warm-up– make first questions easy and non-threatening. • Main body – present questions in alogicalorder • A cool-off period – includea few easy questions to defuse tension at the end • Closure – thank interviewee, signal the end, e.g, switch recorder off.

  16. Enriching the interview process • Props - devices for prompting interviewee, e.g., a prototype, scenario

  17. Questionnaires • Questions can be closed or open • Closed questions are easier to analyze, and may be done by computer • Can be administered to large populations • Paper, email and the web used for dissemination • Sampling can be a problem when the size of a population is unknown as is common online

  18. Questionnaire design • The impact of a question can be influenced by question order. • Do you need different versions of the questionnaire for different populations? • Provide clear instructions on how to complete the questionnaire. • Strike a balance between using white space and keeping the questionnaire compact. • Decide on whether phrases will all be positive, all negative or mixed.

  19. Question and response format • ‘Yes’ and ‘No’ checkboxes • Checkboxes that offer many options • Rating scales • Likert scales • semantic scales • 3, 5, 7 or more points? • Open-ended responses

  20. Encouraging a good response • Make sure purpose of study is clear • Promise anonymity • Ensure questionnaire is well designed • Offer a short version for those who do not have time to complete a long questionnaire • If mailed, include a stamped addressed envelope • Follow-up with emails, phone calls, letters • Provide an incentive • 40% response rate is high, 20% is often acceptable

  21. Advantages of online questionnaires • Responses are usually received quickly • No copying and postage costs • Data can be collected in database for analysis • Time required for data analysis is reduced • Errors can be corrected easily

  22. Problems with online questionnaires • Sampling is problematic if population size is unknown • Preventing individuals from responding more than once • Individuals have also been known to change questions in email questionnaires

  23. Observation • Direct observation in the field • Structuring frameworks • Degree of participation (insider or outsider) • Ethnography • Direct observation in controlled environments • Indirect observation: tracking users’ activities • Diaries • Interaction logging

  24. Structuring frameworks to guide observation • - The person. Who? - The place. Where?- The thing. What? • The Goetz and LeCompte (1984) framework:- Who is present? - What is their role? - What is happening? - When does the activity occur?- Where is it happening? - Why is it happening? - How is the activity organized?

  25. Ethnography (1) • Ethnography is a philosophy with a set of techniques that include participant observation and interviews • Debate about differences between participant observation and ethnography • Ethnographers immerse themselves in the culture that they study • A researcher’s degree of participation can vary along a scale from ‘outside’ to ‘inside’ • Analyzing video and data logs can be time-consuming • Collections of comments, incidents, and artifacts are made

  26. Ethnography (2) • Co-operation of people being observed is required • Informants are useful • Data analysis is continuous • Interpretivist technique • Questions get refined as understanding grows • Reports usually contain examples

  27. Other Observational Methods • Direct observation in a controlled setting • Think-aloud technique • Indirect observation • Diaries • Interaction logs

  28. Choosing and combining techniques • Depends on • The focus of the study • The participants involved • The nature of the technique • The resources available

  29. Data Gathering Summary • Three main data gathering methods: interviews, questionnaires, observation • Four key issues of data gathering: goals, triangulation, participant relationship, pilot • Interviews may be structured, semi-structured or unstructured • Questionnaires may be on paper, online or telephone • Observation may be direct or indirect, in the field or in controlled setting • Techniques can be combined depending on study focus, participants, nature of technique and available resources

  30. Part 3: Designing for Conversation and Coordination • Conversational mechanisms • Coordination mechanisms • Awareness mechanisms • Examples of technologies designed to extend how people • talk and socialise • work together • play and learn together

  31. Conversational mechanisms • Various mechanisms and ‘rules’ are followed when holding a conversation, e.g. mutual greetings A: Hi there B: Hi! C: Hi A: All right? C: Good, how’s it going? A: Fine, how are you? C: OK B: So-so. How’s life treating you?

  32. Conversational rules • Sacks et al. (1978) work on conversation analysis describe three basic rules: Rule 1: the current speaker chooses the next speaker by asking an opinion, question, or request Rule 2: another person decides to start speaking Rule 3: the current speaker continues talking

  33. Conversational rules • Turn-taking used to coordinate conversation • A: Shall we meet at 8? • B: Um, can we meet a bit later? • A: Shall we meet at 8? • B: Wow, look at him? • A: Yes what a funny hairdo! • B: Um, can we meet a bit later? • Back channeling to signal to continue and following • Uh-uh, umm, ahh

  34. More conversational rules • Farewell rituals • Bye then, see you, yer bye, see you later…. • Implicit and explicit cues • e.g., looking at watch, fidgeting with coat and bags • explicitly saying “Oh dear, must go, look at the time, I’m late…”

  35. Breakdowns in conversation • When someone says something that is misunderstood: • Speaker will repeat with emphasis: A: “this one?” B: “no, I meant that one!” • Also use tokens: Eh? Quoi? Huh? What?

  36. What happens in technology-mediated conversations? • Do same conversational rules apply? • Are there more breakdowns? • How do people repair them for: • Phone? • Email? • IM? • Texting?

  37. Synchronous computer-mediated communication • Conversations are supported in real-time through voice and/or typing • Examples include video conferencing, VOIP, MUDs and chat • Benefits include: • Not having to physically face people may increase shy people’s confidence • Allows people to keep abreast of the goings-on in an organization without having to move from their office • Problems: • Difficult to establish eye contact with images of others • People can behave badly when behind the mask of an avatar

  38. Asynchronous computer-mediated communication • Communication takes place remotely at different times • Email, newsgroups, texting • Benefits include: • Read any place any time • Flexible as to how to deal with it • Can make saying things easier • Problems include: • FLAMING!!! • Message overload • False expectations as to when people will reply

  39. Coordination mechanisms • When a group of people act or interact together they need to coordinate themselves • e.g., playing football, navigating a ship • They use: • verbal and non-verbal communication • schedules, rules, and conventions • shared external representations

  40. Verbal and non-verbal communication • Talk is central • Non-verbal also used to emphasize and as substitute • e.g., nods, shakes, winks, glances, gestures and hand-raising • Formal meetings • explicit structures such as agendas, memos, and minutes are employed to coordinate the activity

  41. Schedules, rules and conventions • Schedules used to organize regular activities in large organizations • Formal rules, like the writing of monthly reports enable organizations to maintain order and keep track • Conventions, like keeping quiet in a library, are a form of courtesy to others

  42. Shared external representations • Common method used to coordinate collaborative activities, • e.g., checklists, tables, to-do lists • They can provide external information on: • who is working on what • when it is being worked on • where it is being worked on • when a piece of work is supposed to be finished • whom it goes to next

  43. Collaborative technologies to support coordination • There are a variety of software tools designed to support scheduling, planning and coordinating • e.g., group calendars, electronic schedulers, project management tools, and workflow tools • Need to get balance between human and system control • too much system control and the users will rebel • too little control and the system breaks down

  44. Awareness mechanisms • Involves knowing who is around, what is happening, and who is talking with whom • Peripheral awareness • Keeping an eye on things happening in the periphery of vision • Overhearing and overseeing - allows tracking of what others are doing without explicit cues

  45. Lo tech awareness mechanism

  46. Designing technologies to support awareness • Provide awareness of others who are in different locations • Early example was media spaces “extend the world of desks, chairs, walls and ceilings” (Harrison et al, 1997) • Examples: Clearboard and Portholes

  47. Portholes (Xerox PARC) Regularly updated digitized images of people in their offices appeared on everyone’s desktop machines throughout day and night

  48. Notification systems • Users notify others as opposed to being constantly monitored • Provide information about shared objects and progress of collaborative tasks • examples: Tickertape, Babble

  49. Elvin • Elvin is a distributed awareness system that provides a range of client services (Segall and Arnold, 1997) • It includes Tickertape, one of the first lightweight messaging systems

  50. Conversation and Coordination Key Points • Social mechanisms, like turn-taking, conventions, etc., enable us to collaborate and coordinate our activities • Keeping aware of what others are doing and letting others know what you are doing are important aspects of collaborative working and socialising • Many collaborative technologies systems have been built to support collaboration

More Related