Analysing Ethical Scenarios

1. Analysing Ethical Scenarios From Blaise Liffick CA640

2. Background • In its 1991 Computing Curriculum report, the Association for Computing Machinery (in conjunction with the Institute of Electrical and Electronic Engineers) identified for the first time the importance of including a "social and professional context" as part of the core curriculum for all computer science majors [Tucker, et al, 1991]. According to this report, "undergraduates also need to understand the basic cultural, social, legal, and ethical issues inherent in the discipline of computing" [page 11]. CA640

3. Cases • There are several techniques that have been employed in order to present these topics to students. The most popular technique appears to be what is variously called the case study or ethical scenario, a short narrative of one or more events that involve one or more ethical issues. The intent is that a student reading such a scenario is expected to analyze the participants' actions and arrive at a judgment concerning their ethical nature. CA640

4. The Methodology • The following set of guidelines allows students to dissect a scenario, place the issues identified into a social context, and take advantage of available tools (e.g., various codes of ethics) for analyzing the ethical issues presented. This approach is analogous to a classical computer programming methodology known as top- down analysis or step-wise refinement. The idea of this programming technique is to break a large problem into successively smaller (and, presumably, more easily solved) problems, until solutions to these sub-problems become obvious. The total solution to the problem is then constructed by carefully recombining these smaller solutions. CA640

5. Composition • While students have undoubtedly been taught an outlining method for developing an essay in various writing courses, this method is one of composition, i.e., of putting pieces together in their proper order. CA640

6. Decomposition • The task of critical analysis, however, begins with decomposition, the breaking down of an idea expressed in prose into its component parts. The essential ideas and issues must be identified. The purpose of the proposed methodology is to assist students in that decomposition. Once this has been accomplished, the student can then use traditional writing techniques to develop an essay in which the student presents his or her opinions about the scenario, along with arguments that can be supported with material from the text of the scenario itself. CA640

7. Interactive Human Judgment in a Life-Critical System (Parker [1977]) • The spokesperson for a union of airline maintenance workers charges that the airline has introduced a computer program to perform functions which should require interactive human judgment if safety is to be ensured. The program is one which schedules maintenance, and which reassigns aircraft when emergencies arise because airplanes unexpectedly become unusable. The systems analyst, under whose direction the program was written, is aware that not all operational factors have been taken into consideration in the program, but he had been assured by management that the decision rules used in the program conform to all the requirements of the IA T A (International Air Transport Association). CA640

8. Case cont’d • In his opinion the program should have been an interactive one, where a person is involved in some of the final decision making, but the company was not prepared to go to the additional expense of an interactive system. When testing his program, he could not devise an example where the existing program produced an action which failed to meet a safety condition. Because he could not document reasons for his doubts, and also in part because he was inclined to be defensive about his own work, when he was asked to testify in an inquiry dealing with the union IS complaint, he did not volunteer his opinion on how the system should have been designed. CA640

9. Scenario 1. The following steps assist the student in analysing such a scenario: List Participants and Their Actions CA640

10. List Participants and Their Actions • Make a list of all of the participants involved, and their respective actions. This list should include the following groups: primary participants, those who have taken specific, obvious actions; secondary participants, those who have been acted upon or otherwise been affected by the actions of the primary participants, or who are mentioned in the scenario but take no direct action themselves; and implied participants, those who are not specifically identified by name, but who may have a stake in the outcome of the events described by the scenario - implied participants may be primary or secondary in nature. CA640

11. In Scenario 1, the following participants and their actions: Primary Participants: Systems analyst designed program knowing that not all operational factors had been taken into account informed management of his concerns failed to volunteer his opinion when testifying Airline management disregarded analyst's concerns indicated decision rules used in program conform to I A T A requirements opposed making system interactive CA640

12. Participants (2) Union spokesperson charges airline with ignoring safety factors Union files complaint against airline Inquiry board holds hearing CA640

13. Participants (3) Secondary Participants: Maintenance workers must use the new maintenance software IATA set requirements for airplane maintenance Implied Participants: Programming staff (primary) implemented systems analyst's design Airplane crews and passengers (secondary) must travel in planes maintained by developed system CA640

14. Reduce List through Simplifying Assumptions Many participants are of minimal interest in the analysis of this scenario. Their actions are either trivial, or are not of direct concern ethically. It is important to weed out those who have limited impact on the real issues addressed in the scenario. So that time is not spent analysing people or issues that are not truly of interest. CA640

15. Reduction • For instance, it can perhaps be assumed that the union acted in good faith when filing charges against the airline, that they were not engaged in grandstanding for the sake of publicity, nor were they attempting to harass management during contract negotiations. If the union's complaint is legitimate, we can ignore their action in this matter, and need not comment on it in the final written analysis of the scenario. • Other simplifications include: • There is no distinction between the union spokesperson and the union. • The inquiry board is legitimate, and will do its job properly. • The I A T A has established sound maintenance requirements for all aircraft. The union represents the maintenance workers, and while there is a distinction between these groups to some extent, the distinction is unimportant in this scenario. CA640

16. Reduction (2) • While the main issue appears to be the overall safety of the crews and passengers, this is stating the obvious and can probably be eliminated from the list. • While a defense of “I just followed orders” is hardly legitimate in many cases, the programming team who did the actual implementation can probably be eliminated from consideration, on the assumption that they followed standard programming practices in coding the programs for the system. Another consideration is that it was the systems analyst who was in charge of the outcome of the project, and who, therefore, developed the specifications that the programmers implemented. CA640

17. Final reduction • As a result of these assumptions, all the participants except the systems analyst and the airline's management can be eliminated. While this reduction was perhaps obvious in a certain respect, it was necessary to go through the steps of including all of the other participants, and then eliminate them, for the sake of completeness. It is all too easy to assume that those who are mentioned most prominently in a scenario are the only ones who have an impact on the final analysis. CA640

18. Legal Considerations • Certainly, if an action is against the law, it is probably unethical. List what laws cover the actions discussed in the scenario (clearly, students must be introduced to the pertinent laws regarding privacy, hacking, etc.). Obviously, students are not expected to be as knowledgeable as lawyers about all aspects of applicable law. However, they should at least be aware of the major laws directly related to their field. • A subset of legal considerations is company policy. There may be no current laws that govern a particular action, but there may be a company policy that requires or prohibits certain actions. Such policies must be explicitly stated in the given scenario -they should never be assumed. • Also, just because there is not a law or policy prohibiting an action, this does not mean that the action is ethical. CA640

19. Legal Considerations (2) • In the example scenario, there are likely to be many laws pertaining to the regulation of the airline industry. The I A T A must have some authority in the matter, since it has developed a set of maintenance requirements. It is impossible, however, to analyze this (probably lengthy and numerous) set of laws that might have bearing in this case. One must assume that any violation of law will be determined by the board of inquiry. • A legal issue that would be pertinent to this case, however, would be liability law. What liability does the airline management assume in this case, should an airplane crash due to faulty maintenance? the systems analyst? the programming staff? the maintenance workers? • While, again, it would take a legal specialist a great deal of time to present the entire set of laws and legal decisions that affect the case, it is clearly important to provide some guidance to students on this issue. Clearly life-critical systems present special problems of liability. Students can at least be made aware that liability is an issue that they should discuss with their employers and, perhaps, a good attorney. CA640

20. List Possible Options of the Participants • Make a list of what options the participants may have had before they chose the path of action described in the scenario. While this list could potentially be infinite (e.g. "the participant could have committed suicide..."), clearly only pertinent and viable options need be listed. The scenario may not list the options faced by the participant. In that case, the options are implied. Obviously, if the participant chose Action A, Action B (or C, etc.) was not chosen, which must be known if the reader is going to make a judgment of ethical behavior. This list helps ascertain whether or not there existed another alternative for the participant that was purely ethical, or whether the participant perhaps had chosen the least non-ethical option. CA640

21. Options In Scenario 1, the analyst could have • refused to participate in the design unless all operational factors were incorporated • suggested involving the maintenance workers themselves in the design effort • contacted the I A T A directly about his concerns • offered his opinion to the inquiry board • contacted the media about his concerns CA640

22. Options (2) The airline management could have • involved the maintenance workers themselves in the design effort • ensured that all operational factors were incorporated in the design • taken the systems analyst's advice and agreed to allow the system to include an interactive feature CA640

23. List Possible Justifications for the Participants' Actions In order to evaluate why the participants may have chosen their respective response(s) to the situation, compile a list with as many reasons as might be suggested by the scenario. List justifications explicitly offered in the scenario, as well as any other justifications that make sense, being careful not to make assumptions. For instance, when considering possible justifications for the main participant in a scenario, it is easy to suggest a defense if you, yourself, have been in a similar situation and had to make the same kind of decision. CA640

24. Justifications (1) In this scenario, the analyst may justify his actions (or, in this case, non-action) by stating that he • simply followed a "management decision." • couldn't prove there was a problem, even with additional testing. • had only his own professional opinion (which might be taken simply as a "hunch ") that the system was deficient. CA640

25. Justifications (2) The airline management might state that • the cost of redesigning the system for interactive use is prohibitively expensive, and could put the company in a financially precarious position. • they do in fact meet the I A T A requirements, and that there is no reason to be any more stringent in their maintenance processes. • it is unreasonable to expect the company to base an expensive redesign on a single employee's "hunch," however informed or expert the person is. • It is important that this list be limited to legitimate justifications, and not be simply a compilation of rationalizations (unless they are somehow explicitly stated in the scenario). CA640

26. List Key Statements • List quotations from the text that are important to the analysis. These will probably have been used to create parts of the previous lists, and so should be mostly obvious. Jot down any phrase that is used as the basis for previously listed items. • These phrases might follow an action and begin with an explicit or implied 'even though,' 'however,' or 'instead of.' Other important phrases include those that indicate secrecy, such as 'without getting approval' or 'without telling anyone.' Unmistakable indicators are those expressions that suggest the participants' refusal to accept responsibility for their actions, statements where the participants make excuses for their behavior, or statements that demonstrate personal gain for the main participant. CA640

27. For this scenario, the following phrases are important: • "...not all operational factors have been taken into consideration..." • "...assured by management that the decision rules used in the program conform to all the requirements of the I A T A." • "...the company was not prepared to go to the additional expense..." • "...he could not devise an example where the existing program produced an action which failed to meet a safety condition." • "...he did not volunteer his opinion on how the system should have been designed." CA640

28. List Questions Raised • According to Parker (1979), the reader should not make assumptions regarding what else might have happened in the scenario. He warns against assuming that just because one thing has happened, we can infer the circumstances leading up to that event. He instructs the reader to rely strictly on what was included in the scenario because of excess baggage that might be carried into the analysis by the reader. Otherwise what may follow is a lengthy deliberation exploring extensions of the given scenario, which may result in the discussion of a situation that only vaguely resembles the original scenario. • However, natural language is ambiguous by its very nature. N o matter how carefully a particular scenario might have been crafted, and regardless of exhortations to rely solely on what is directly supportable by the text, there are few instances where something isn't ambiguous about a given scenario. In addition, there are often simply unknowable pieces of information that have a direct bearing on the outcome of a scenario. CA640

29. Were standard design and programming practices followed in this project? There is no information available within the scenario to assist in answering this question. One could argue that a great deal hinges on the answer to this one question, however. It must be considered that such ambiguities point to a fundamental flaw in the scenario, i.e. that the author did not craft the case carefully enough. One would think that an important goal in developing a scenario would be to make it entirely unambiguous. The nature of natural language, however, makes this a nearly impossible task. Other questions that might be raised with the example scenario include • Was the issue of including representatives of the maintenance crew on the design team ever brought up? Was there a conscious decision to exclude them? • Could a cost/benefit analysis have been done to demonstrate the efficacy of an interactive system? What about a prototype? . CA640

30. Questions (2) • Should a computer ever be given sole authority in matters where human lives are clearly at risk? • This last question, of course, is widely open-ended, and is ultimately the focal point of the discussion. • Real life is full of ambiguities. The vagaries of natural language make ambiguity in nearly any set of statements inevitable. Although human beings are generally adept at dealing with ambiguity on a regular basis, clearly there would not be a need for the extensive legal system of our modem society if we all agreed on interpretations of events, statements, etc. It is necessary to at least acknowledge that ambiguities exist even in the most carefully crafted statements. Thus it is necessary to perform some amount of exploration into the various possible interpretations that a scenario might evoke from multiple readers CA640

31. Other Models, Related Issues • Scenarios in computer science may appear to be unique, but when broken into pieces or distilled into actions irrespective of the hardware or environments involved, may be similar to other real world systems for which ethical issues have largely been resolved. These models can serve as analogies for a given situation. Once the relationship of two seemingly dissimilar scenarios is made obvious, a conclusion may be easier to draw. • Using the example of electronic mail, we can look at the models presented by usage of telephones, the postal system, or inter-office mail. When we ask, 'does a company have the right to monitor electronic mail,' we might ask the similar question, 'does an employer have the right to monitor employee phone calls, open employee letters or read someone's inter-office memo?' Questions like these may have been answered quite emphatically for the existing models, and could be applied to the seemingly new, yet age-old question raised by computer-related application. This is similar to the notion of homomorphism in discrete mathematics. CA640

32. Analogies • Nearly any life-critical system might serve as an analogy for the example scenario. The Therac-25 disaster certainly comes to mind. Another is the O-ring incident of the Challenger space shuttle. One other analogy that might come to mind would be the development of the Mercury space capsule. After input from the original seven astronauts (who were intended initially to be just passengers, not pilots), the capsule was heavily modified to include on-board flight controls. Clearly in this instance, it became crucial to involve those most immediately affected by an automated system, the direct user CA640

33. Comparison to Codes of Ethics For the example, at least the following ACM principles apply: • 1.1 - Contribute to society and human well-being. • 1.2 - Avoid harm to others. • 1.3 - Be honest and trustworthy. • 2.1 - Strive to achieve the highest quality... [in your] professional work. • 2.5 - Give comprehensive and thorough evaluations of computer systems and their impacts, including analysis of possible risks. • 3.2 - Manage personnel and resources to design and build information systems that enhance the quality of working life. • 3.4 - Ensure that users...have their needs clearly articulated during the assessment and design of requirements... CA640

34. The Place of Ethical Theory • A more significant reason for not including ethical theories in this methodology is, however, that it would be redundant to do so. The codes of ethics and conduct of professional organizations already incorporate a particular point of view about the nature of ethics as applied to that profession. While these codes might not satisfy a philosopher in either form or content, they are nonetheless the expression of that profession's position on ethics. • Historically such codes have often been viewed as somewhat superfluous, as little more than flowery sentiment, with no practical aspect to them. This methodology points out, however, that these codes can indeed be put to practical use. CA640

35. Composition • Once the student follows this methodology to completion, the task of decomposition is over and the student must now reassemble the parts into a coherent essay, i.e., must construct a composition. The student has numerous small parts on which to draw for this composition process. It is now up to the student to select the most important items from the various lists to discuss in his or her essay. • One caveat, however, is that students tend to view the decomposition methodology as an outline for their final essay. Without additional directions on how to develop the essay itself, they frequently will simply start at step 1 of the process, write about the information in their list for a few paragraphs, then move to the step 2 list, write a short time, etc. until they reach the end of the steps. While this may produce a coherent presentation of the information, it also creates rather uninteresting essays, all sounding essentially the same (whether this is a positive thing is a moot point). Students must typically be encouraged to use their own creativity to liven up the final review and its presentation of the facts. CA640

36. Conclusions • A reason for student dislike of ethical assignments is the open-ended nature of the task itself. Most students seem to prefer dealing with questions that have a clearly definable "right" answer. The methodology presented in this paper is in no way an attempt to come up with a process that produces a single answer to an ethical dilemma that can be considered "right." However, students have been almost universally trained to expect such an outcome, not only from their science classes ( where experiments clearly have only one correct solution), but also their social science and humanities classes, where the type of assignments and tests that are given rely heavily on questions that the student is expected to answer in a particular way. CA640

37. Conclusions (2) • The quality of papers has risen significantly since this methodology has been introduced, especially in the case of marginal students. In addition, this methodology actually does provide a way for the professor to defend what would normally appear to be a purely subjective grade. If a student has omitted certain aspects of the analysis, the professor can point to any area that is lacking and suggest that the student overlooked one aspect or another, hence the resulting grade. In addition, the student's analysis itself could be graded for completeness, as well as used as a checklist when grading the final essay. • One additional benefit to this methodology is that it parallels a process with which the students are already familiar, namely top-down structured programming. Not only does this make this new process more understandable to the student, but it provides a concomitant strengthening of their understanding of the top-down methodology they use in programming tasks as well. CA640