1 / 16

Ethics and Software Reliability: Understanding Bugs and Verification Activities

This article explores the relationship between ethics and software reliability, discussing common errors in design and implementation, as well as the challenges of verification and validation. It also examines real-life examples of reliability failures and poses ethical questions regarding moral responsibility and warranties in the software industry.

nadel
Download Presentation

Ethics and Software Reliability: Understanding Bugs and Verification Activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ethics and Software Reliability http://flic.kr/p/doFGYA

  2. How we catch bugs: V & V activities • Verification answers: Did we build the system right? • Errors in design (wrt the requirements spec)? • Errors in implementation (wrt the requirements spec)? • Validation answers: Did we build the right system? • Errors in the requirements spec?

  3. How might we verify/validate a program? • Prove correct (w/ formal math)? • Nope. Too difficult, even for some small programs • Warm(?) research area: Logical inference • Prove a model correct? • Nope. Too difficult to get small enough model that captures all relevant behavior • Hot(?) research area: Model checking • Run the program to see if it works? • Yup. That’s what we do • Hot area of research and practice: Testing http://flic.kr/p/9ksxQa

  4. The Testing Problem:How to choose small set of test casesthat reveal all errors? • Fundamental research problem • Essentially unsolvable in general case

  5. Does all “real” software contain bugs? Yup.No software is 100% reliable Let’s consider some examples of reliability failures http://flic.kr/p/9ksxQa

  6. Financial software failures • Knight Capital loses $440 mil.: http://www.npr.org/2012/08/18/159082822/in-wall-street-2-0-computers-are-king • L.A. teachers’ pay: http://www.npr.org/templates/story/story.php?storyId=14869684 • Phone bill for $57,346.20 (Minnesota, 2001) • Beef producers lose $15-$20 million due to bad price reports

  7. Data entry/retrieval failures • See this story: http://www.nbc.com/news-sports/msnbc-video/2012/11/rossen-reports-background-check-mistakes-cost-jobs/ • Thousands of voters forbidden to vote because computer miscategorized them as felons (Florida, 2000) • Numerous false arrests due to errors in crime database (NCIC)

  8. Software system failures • Mars Surveyor failure (2007): http://www.npr.org/templates/story/story.php?storyId=6817176 • Therac-25 (1985-1987): multiple radiation overdoses/deaths • Patriot missile system: 28 soldiers killed by SCUD; Patriot never even fires (1991) • Ariane 5 rocket: reuse error causes self-destruct, costing $500 mil.

  9. Software unreliability leads tomoral and ethical questions • Morality: Rules of conduct regarding what people ought and ought not do in various situations • Different societies may have different morals • Ethics: Rational examination of people’s moral beliefs and behaviors • Done by reasoning from facts to commonly accepted values • Ethical theories: frameworks for reasoning • Examples: Subjective Relativism, Cultural Relativism, Divine Command Theory, Ethical Egoism, Kantaniasm, Utilitarianism…

  10. Morality and Ethics From Quinn (2013), p53

  11. Ethical Question Amazon error (UK, 2003): iPaq price listed as 7 EUR instead of 275 EUR Q: Amazon refuses to honor sales—were they wrong? • Utilitarian view: Seller must always honor advertised price • Kantian view: Consumers recognized error and did not act in “good faith”

  12. Ethical Question Should Therac-25 developers be held morally responsible for deaths? • Moral responsibility requires: • Causal condition: actions caused harm • Mental condition: actions intended • Unintended harm counts if due to carelessness, recklessness, or negligence • Quinn concludes “yes” because machine lacked devices to catch/prevent overdose and failed to thoroughly verify overdose wasn’t possible

  13. If perfect software is impossible, what sort of warranties should companies provide? • Shrinkwrap warranties: • 90-day replacement (haha)/money back if softw. fails • Few guarantees (you’ll be able to install it) • Maker has no liability (for harm to your business) • “Don’t blame us if it’s full of bugs” • Are warranties enforceable? • Unfair warranties could violate laws • Court rulings have gone both ways • Should manufacturers be held liable for defects? • Should consumers have to pay for bug fixes?

  14. Hypothetical Question • Medick Inc. developing fancy new mobile software for nurses and other hospital workers • Medick told customers it would be ready next week • Software still contains lots of bugs • Competitor will release new product in a few weeks • If competitor gets to market first, Medick will likely go under What course of action should Medick take? • Perspectives: software engineers, sales force, managers, venture capitalists, and nurses

  15. Summary • All “software” contains bugs • Knowing that perfection is impossible, how to set consumer expectations and assign blame when things go wrong? • Leads to moral and ethical questions http://flic.kr/p/YSY3X

  16. Question • Anyone been a beta tester? • What did it teach you about computer reliability?

More Related