Impact of Computers on Society

Impact of Computers on Society 4. Can We Trust the Computer?

Two Main Areas of Concern • Reliability • Accuracy • Combined with… • Hardware • Software

Some Failures • AT&T phone system shutdown, January 1990 • Subsequent smaller outages • Caused by software flaws, hardware changes • Airport baggage systems in Denver and Hong Kong • CTB/McGraw-Hill educational testing • FBI abandons criminal tracking DB, 2005 • IRS • FAA Air Traffic Control • Strategic Defense Initiative • Two major new systems at NSA do not work (2006) • NSA nearly overloads local power grid (2007)

Have you experienced a failure? • Your own computer • A business system • An educational system • A governmental system

Program Correctness • Ignore hardware for the moment • Can you guarantee program correctness? • Fairly easy to prove incorrectness • Theoretically impossible to prove correctness • A simple program with just 10 branches or decisions would require 10! tests – that’s 3,628,800 • Exhaustive testing can only improve the odds

Three levels of failure • Individual end user or consumer -- affects only that person • System failure – affects many, perhaps millions • Critical failure – may result in loss of life

Failures: Databases • “Computer error” is generally human error • Law enforcement databases • Not up to date • Inaccuracies • Sometimes presume guilt • City of Frederick Maryland Police • Sen. Edward Kennedy and others flagged by the air travelers No-Fly DB. • National Crime Information Center

Failures: Systems • Communications (AT&T already mentioned) • Business, financial, transportation • FBI and others have abandoned systems • System must be able to handle errors • Question any sudden change in output • James Stewart, nuclear engineer – non-linear errors • The problem of computational complexity • A host of sources of errors, Baase p. 155 • MSD – too cheap to buy a backup drive • WordPerfect for DOS—floppy disc full

Failures: Software Reuse • The Object-Oriented programming paradigm • Libraries of pre-coded reusable components • Should increase reliability, but not always… • Designed for one environment, but used in another • The Ariane 5 rocket: overflow

Overflow/Underflow: an example • A byte has 8 bits, which equals 256 • Computers count from 0 to N-1 • What is the next number? • 000 = 0000 0000 • 001 = 0000 0001 • 002 = 0000 0010 • 003 = 0000 0011 • And so forth… • 253 = 1111 1101 • 254 = 1111 1110 • 255 = 1111 1111 • 256 = ?? • 256 = 1 0000 0000

Failures: User Interfaces • An entire course titled HCI • Think of a device you hate… • Therac-25 • The max/min/close buttons in MS Win vs. Mac OS X • Gauges in aircraft in WWII • Inconvenient menu structures

Preventive Measures • Exhaustive testing • Remember that you can never prove there are no flaws • Unforeseen problems and flaws may emerge later • Over-engineering: design for greater capacity than needed • Systems redundancy: the US space shuttle

Some protection against failure • Legal recourse • Very expensive for both parties • Can put a firm out of business • Warranties • “Shrink-wrap” warranties: very one-sided • Sometimes difficult to get satisfaction

Additional protections • Regulation • Requires an agency to oversee • Regulations are usually written after the fact • Slows the introduction of new technology: expensive and time-consuming • Sometimes affected by political concerns: influence of large donors

More Protection • Professional licensing • CS is not yet a true profession • An accrediting organization (ACM, IEEE) • A competency exam or apprenticeship • Requirement for continuing education • A recognized license • An enforced code of ethics • An adjudication process

Somebody Must Take Responsibility • Responsibility shared between consumer/user, the vendor/integrator, and the company/developer • RTM? • Make realistic claims about the technology • Provide a clear means of communication • An area still in flux

Put Things in Perspective • It’s not all catastrophic – some errors are minor • You can’t guarantee anything 100% of the time • Some problems are old ones in a new guise • What is an acceptable error rate? • Consider how much damage was done? • A document that didn’t print correctly • A massive failure involving thousands of individuals and businesses • Injury or loss of life

How Dependent Are We? • VERY! • Can you think of something you used today that did not involve a computer at some stage of its design, manufacture, or use?

Computer Models • A mathematical model of a real-world process • Link Trainers in WWII • Modern aircraft trainers • Population growth • Vehicle accident simulation • Climate models • “Chaotic” systems • The “butterfly effect” – a small change in input results in a large difference in output

Limits on Models • Computational complexity • Unknown factors • Use of approximation • Incomplete data • Incorrect data

Four Considerations when Creating a Computer Model • How well is the process (the thing to be simulated) understood? • How good are the data? • What simplifications have been made? • If possible, how do the results compare with observed reality?

Don’t exaggerate • People have a tendency to exaggerate • People have a tendency to play down negative aspects • If it sounds too good to be true…

Impact of Computers on Society