Computing and Ethics

1. Computing and Ethics Some of these slides are derived from Sherry Clark, A Gift of Fire; Prof. John Nestor, Lafayette College; Russell Gayle, UNC; H. Scott Matthews, Carnegie Mellon University, Michael J. Quinn, Ethics for the Information Age

2. What is Ethics? Ethics – “philosophical study of morality” Philosophy -- Investigation of the nature, causes, or principles of reality, knowledge, or values, based on logical reasoning rather than empirical methods.

3. What is Ethics? Ethics – “philosophical study of morality” Morality – rules of conduct describing what people in a society should and should not do Association of people organized under a system of rules designed to advance the good of its members over time Rawls, A Theory of Justice

4. Ethical Dilemmas • Apparent conflict between moral imperatives • New technologies can open up new social problems and new ethical dilemmas • Examples? • Can new technologies change morality? • Oz’s cycle of change: • Technological change => Ethical pronouncements => Discussion => Laws => (back to Tech. Change)

5. Studying Ethics • Two Different Approaches • Descriptive ethics: what people believe to be right and wrong • Normative ethics: what people should believe is right and wrong • Relationship between normative and philosophical ethics? • Examples where descriptive and normative ethics differ?

6. Scenario 1 • You are the senior software engineer at start-up developing software for handheld computers to help nurses keep track of patients • Sales force has promised product by next week • Product still contains many minor bugs • No major bugs have been found, but QA recommends another month of testing • A competitor plans to release a similar product in a few weeks • If your product is not first to market your start-up will probably go out of business

7. Scenario 1 • Should you recommend release of the product next week? • Who will benefit if the company follows your recommendation? • Who will be harmed if the company follows your recommendation? • Do you have an obligation to any group of people that may be affected by your decision?

8. Subjective Relativism • Relativism in General • No universal norms of right and wrong • One person can say “X is right,” another can say “X is wrong,” and both can be right • Subjective relativism • Each person decides right and wrong for himself or herself • “What’s right for you may not be right for me”

9. Subjective relativism evaluation of spam • Spammers say spam is good • Spam brings advertisements to the attention of some people who want to buy their products • Spammers make money • Purchasers are happy to buy their products • Most spam recipients and ISPs say spam is bad • Spam wastes time and computer resources, congests networks, slows processing of non-spam email

10. Subjective Relativism • Pros • Well-meaning and intelligent people disagree on moral issues • Ethical debates are disagreeable and pointless • Cons • Blurs distinction between doing what you think is right and doing what you want to do • Makes no moral distinction between the actions of different people • SR and tolerance are two different things • Decisions may not be based on reason • Not a workable ethical theory

11. Cultural Relativism • What is “right” and “wrong” depends upon a society’s actual moral guidelines • These guidelines vary from place to place and from time to time • A particular action may be right in one society at one time and wrong in other society or at another time

12. Case for Cultural Relativism • Different social contexts demand different moral guidelines • It is arrogant for one society to judge another • Morality is reflected in actual behavior

13. Case Against Cultural Relativism • Because two societies do have different moral views doesn’t mean they ought to have different views • Doesn’t explain how moral guidelines are determined • Doesn’t explain how guidelines evolve • Provides no way out for cultures in conflict • Societies do, in fact, share certain core values • Only indirectly based on reason • Not a workable ethical theory

14. Kantianism • Good will: the desire to do the right thing • Immanuel Kant: Only thing in the world good without qualification is good will. • Reason should cultivate desire to do right thing.

15. Categorical Imperative (1st Formulation) Act only from moral rules that you can at the same time will to be universal moral laws.

16. Illustration of 1st Formulation • Question: Can a person in dire straits make a promise with the intention of breaking it later? • Proposed rule: “I may make promises with the intention of later breaking them.” • The person in trouble wants his promise to be believed so he can get what he needs. • Universalize rule: Everyone may make & break promises • Everyone breaking promises would make promises unbelievable, contradicting desire to have promise believed • The rule is flawed. The answer is “No.”

17. Categorical Imperative (2nd Formulation) Act so that you treat both yourself and other people as ends in themselves and never only as a means to an end. This is usually an easier formulation to work with than the first formulation of the Categorical Imperative.

18. Kantian evaluation of spam (1st formulation) • Proposed rule: • I can send advertisements to as many email addresses as I want • Spammers want people to read their email and buy their products • Universalize rule: • Everyone can send advertisements to as many email addresses as they want • Consequence • If everyone sent advertisements to as many email addresses as they wanted to, email would be so clogged with spam that it would no longer be useful and people would stop using it • The rule is flawed -> spamming is not ethical

19. Kantian evaluation of spam (2nd formulation) • Spammers send ads for a product to many people, knowing only small number will be interested • Most message recipients will waste time and money • Spammers do not respect recipients’ time or money, and are only interested in using spam recipients to make a profit • Thus spammers treat recipients as means to an end • Conclusion: Spamming is wrong

20. Case for Kantianism • Rational • Produces universal moral guidelines • Treats all persons as moral equals • Workable ethical theory

21. Case Against Kantianism • Sometimes no rule adequately characterizes an action. • There is no way to resolve a conflict between rules. • Kantianism allows no exceptions to moral laws.

22. Utilitarianism • Jeremy Bentham and John Stuart Mill • An action is good if it benefits someone • An action is bad if it harms someone • Utility: tendency of an object to produce happiness or prevent unhappiness for an individual or a community

23. Principle of Utility An action is right (or wrong) to the extent that it increases (or decreases) the total happiness of the affected parties. “Greatest Happiness Principle”

24. Act Utilitarianism • Utilitarianism in General • Morality of an action has nothing to do with intent • Focuses on the consequences • A consequentialist theory • Act utilitarianism • Add up change in happiness of all affected beings • Sum > 0, action is good • Sum < 0, action is bad

25. Act utilitarian evaluation of spam • Spam sent to 100 million people • 1 in 10,000 buy product • 90% of people who buy product are happy with it, other 10% feel ripped off • People who don’t buy product waste time and money, get annoyed, etc. - unhappy • Spammer makes lots of money and is VERY happy • 9001 happy people, 99,990,000 unhappy people • Conclusion: 99.991% of people are unhappy, so spam is wrong

26. Case for Act Utilitarianism • Focuses on happiness • Down-to-earth (practical) • Comprehensive • Workable ethical theory

27. Case Against Act Utilitarianism • Unclear whom to include in calculations • Too much work • Ignores our innate sense of duty • Susceptible to the problem of moral luck Sometimes actions have unintended consequences – Moral worth of action is dependent on consequences that may not be under control of moral agent

28. Rule Utilitarianism • We ought to adopt moral rules which, if followed by everyone, will lead to the greatest increase in total happiness • Act utilitarianism applies Principle of Utility to individual actions • Rule utilitarianism applies Principle of Utility to moral rules

29. Anti-Worm Scenario • August 2003: Blaster worm infected thousands of Windows computers • Soon after, Nachi worm appeared • Took control of vulnerable computer • Located and destroyed copies of Blaster • Downloaded software patch to fix security problem • Used computer as launching pad to try to “infect” other vulnerable PCs

30. Evaluation using Rule Utilitarianism • Proposed rule: If I can write a helpful worm that removes a harmful worm from infected computers and shields them from future attacks, I should do so • Who would benefit • People who do not keep their systems updated • Who would be harmed • People who use networks • People who’s computers are invaded by buggy anti-worms • System administrators • Conclusion: Harm outweighs benefits. Releasing anti-worm is wrong.

31. Case for Rule Utilitarianism • Compared to act utilitarianism, it is easier to perform the utilitarian calculus. • Moral rules survive exceptional situations • Avoids the problem of moral luck • Workable ethical theory

32. Case Against Utilitarianism in General • All consequences must be measured on a single scale. • Utilitarianism ignores the problem of an unjust distribution of good consequences.

33. Mail Abuse Prevention System • MAPS is a not-for-profit organization • Contacts marketers who violate MAPS standards for bulk email • Puts marketers who violate standards on a Realtime Blackhole List (RBL) • Some mail relays refer to RBL list • Looks up email host name on RBL list • If name on list, the email gets bounced back • All email from blacklisted hosts gets bounced, even email from non-spammers

34. Was the creation of the RBL ethical? • Utilitarian evaluation: • ISP using RBL benefits by getting better network performance, fewer angry users • But their users are unable to receive email from innocent users of blacklisted ISPs, reducing their utility • Innocent users of blacklisted ISPs unable to communicate with ISPs that user RBL • Conclusion depends on magnitude of benefit and ratio of blacklisted innocent users to total email users • Kantian evaluation: • MAPS puts ISPs on RBL with goal of getting innocent users to complain and pressure ISP to drop spammers • Innocent users are treated as means to an end • This violates Categorical imperative -> RBL is unethical

35. Problem: Therac-25 • The Therac-25 was a software-controlled radiation-therapy machine used to treat people with cancer. • Overdoses of radiation • Normal dosage is 100–200 rads. • It is estimated that 13,000 and 25,000 rads were given to six people. • Three of the six people died.

36. Problem: Therac-25

37. Problem: Therac-25

38. Problem: Therac-25

39. Problem: Therac-25 • Therac-25 Radiation Overdose • Multiple Causes: • Poor safety design. • Insufficient testing and debugging. • Software errors. • Lack of safety interlocks. • Overconfidence. • Inadequate reporting and investigation of accidents. Q: What can be learned from this case?

40. Problem: Ariane 5 • In 1996 Ariane 5 Flight 501 exploded after launch. • Estimated cost of accident: $500 million Video

41. Problem: Ariane 5 • The cause was traced to the Inertial reference system (SRI). • Both the main and backup SRI failed. • Both units failed due to an out-of-range conversion • Input: double precision floating point • Output: 16-bit integer for “horizontal bias” (BH) • Careful analysis during software design had indicated that BH would “fit” in 16 bits • So, why didn’t it fit?

42. Problem: Ariane 5 • Careful analysis during software design had indicated that BH would “fit” in 16 bits • BUT, all analysis had been done for the Ariane 4, the predecessor of Ariane 5 - software was reused • Since Ariane 5 was a larger rocket, the values for BH were higher than anticipated • AND, there was no handler to deal with the exception! http://www.ima.umn.edu/~arnold/disasters/ariane.html Q: What can be learned from this case?

43. Software Engineering Code of Ethics • http://www.acm.org/about/se-code • See “short version” at top of page for general ideas • See “long version” below for more detail