The Economics of Security and Privacy

1. The Economics of Security and Privacy Ross Anderson Cambridge University

2. Background • Economics and security diverged after WW2; started coming back together recently • Economists started thinking about crime and policing in late 60s, about privacy in late 70s • Information security economics started growing five years ago • Many new ideas in last couple of years • Workshop on Economics and Infosec every spring

3. Privacy - First Wave • ‘Right to be left alone’, Brandeis 1890 • Privacy violation as a tort - false light, misappropriation, intrusion (Prosser 1960) • Westin, 1967 - data shadow, privacy as informational self-determination • Inspiration for European data protection movement

4. Privacy - Second Wave • Becker 1968 - economic analysis of crime • Hirshleifer, 70s - conflict theory • Stigler, 1980 - free exchange of information brings Pareto improvement regardless of ownership (bad debtors pay more regardless) • Posner - poor employees want to hide data, good ones to reveal it; privacy inefficient, redistributive • Noam - PETs may change who pays but not what happens - they just redistribute (poor to rich) • Price discrimination is efficient (albeit unpopular)

5. Economics of Information Security • Over the last four years, we have started to apply economic analysis to information security • Economic analysis often explains security failure better then technical analysis! • Information security mechanisms are used increasingly to support business models rather than to manage risk • Economic analysis is also vital for the public policy aspects of security

6. Traditional View of Infosec • People used to think that the Internet was insecure because of lack of features – crypto, authentication, filtering • So engineers worked on providing better, cheaper security features – AES, PKI, firewalls … • About 1999, we started to realize that this is not enough

7. New View of Infosec • Systems are often insecure because the people who could fix them have no incentive to • Bank customers suffer when bank systems allow fraud; patients suffer when hospital systems break privacy; Amazon’s website suffers when infected PCs attack it • Security is often what economists call an ‘externality’ – like environmental pollution • Provides an excuse for government intervention

8. New Uses of Infosec • Xerox started using authentication in ink cartridges to tie them to the printer • Followed by HP, Lexmark … and Lexmark’s case against SCC, and EU Parliament Directives • Motorola started authenticating mobile phone batteries to the phone • BMW now has a car prototype that authenticates its major components

9. IT Economics (1) • The first distinguishing characteristic of many IT product and service markets is network effects • Metcalfe’s law – the value of a network is the square of the number of users • Real networks – phones, fax, email • Virtual networks – PC architecture versus MAC, or Symbian versus WinCE • Network effects tend to lead to dominant firm markets where the winner takes all

10. IT Economics (2) • Second common feature of IT product and service markets is high fixed costs and low marginal costs • Competition can drive down prices to marginal cost of production • This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … • These effects can also lead to dominant-firm market structures

11. IT Economics (3) • Third common feature of IT markets is that switching from one product or service to another is expensive • E.g. switching from Windows to Linux means retraining staff, rewriting apps • Shapiro-Varian theorem: the net present value of a software company is the total switching costs • This is why so much effort is starting to go into accessory control – manage the switching costs in your favour

12. IT Economics and Security • High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant-firm markets with big first-mover advantage • So time-to-market is critical • Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ is not perverse behaviour by Bill Gates but driven by economics • Whichever company had won in the PC OS business would have done the same

13. IT Economics and Security 2 • When building a network monopoly, it is also critical to appeal to the vendors of complementary products • E.g., application software developers in the case of PC versus Apple, or now of Symbian versus CE • Lack of security in earlier versions of Windows makes it easier to develop applications • Similarly, motive for choice of security technologies that dump the support costs on the user (e.g. SSL, PKI, …)

14. Why are many security products ineffective? • Akerlof’s Nobel-prizewinning paper, ‘The Market for Lemons’ provides key insight – asymmetric information • Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000 • What is the equilibrium price of used cars in this town? • If $1500, no good cars will be offered for sale … • Usual fix: brands (e.g. ‘Volvo certified used car’)

15. Security and Liability • Why did digital signatures not take off (e.g. SET protocol)? • Industry thought: legal uncertainty. So EU passed electronic signature law • Recent research: customers and merchants resist transfer of liability by bankers for disputed transactions • Best to stick with credit cards, as any fraud is the bank’s problem • Similar resistance to phone-based payment – people prefer prepayment plans because of uncertainty

16. Why Bill wasn’t interested in security • While Microsoft was growing, the two critical factors were speed, and appeal to application developers • Security markets were over-hyped and driven by artificial factors • Issues like privacy and liability were more complex than they seemed • The public couldn’t tell good security from bad anyway

17. Why is Bill changing his mind? • ‘Trusted Computing’ initiative ranges from TCG and NGSCB to the IRM mechanisms in Office 2003 • IRM – Information Rights Management – changes ownership of a file from the machine owner to the file creator • Files are encrypted and associated with rights management information • The file creator can specify that a file can only be read by Mr. X, and only till date Y • What will be the effect on the typical business that uses PCs?

18. Why is Bill changing his mind? (2) • At present, a company with 100 PCs pays maybe $500 per seat for Office • Remember – value of software company = total switching costs • So – cost of retraining everyone to use Linux, converting files etc is maybe $50,000 • But once many of the documents can’t be converted without the creators’ permission, the switching cost is much higher • Lock-in is the key!

19. Open or Closed? • Free/open source view - easier for defenders to find and fix bugs (‘to many eyes, all bugs are shallow’) • NSA view - easier for attackers to find and exploit bugs • Under standard reliability growth model assumptions, openness helps attackers and defenders equally • Whether open or closed is better will depend on how your system departs from the ideal

20. How often should we patch? • Big topic at WEIS 2004, two weeks ago • Rescorla: bugs independent, most exploits follow patching - so we should never disclose vulnerabilities or ship patches • Arora, Telang, Xu: under different assumptions, we should cut disclosure delay • Arora, Telang et al: some empirical evidence - disclosure increases attacks, patching cuts • Ozment - auction theory may give some ideas

21. How are Incentives Skewed? • If you are DirNSA and have a nice new hack on NT, do you tell Bill? • Tell – protect 300m Americans • Don’t tell – be able to hack 400m Europeans, 1000m Chinese,… • If the Chinese hack US systems, they keep quiet. If you hack their systems, you can brag about it to the President and get more budget

22. Skewed Incentives (2) • Within corporate sector, large companies spend too much on security - small companies too little • Adverse selection effect: the most risk-averse people end up as corporate security managers • More risk-loving people may be sales or engineering staff, or small business entrepreneurs • Also: due-diligence effects, government regulation, insurance market issues • We tolerate attacks on stuff we already know to be useful (smartphone viruses worse than PC viruses)

23. How Much to Spend? • How much should the average company spend on information security? • Governments, vendors: much much more than at present • They’ve been saying this for 20 years! • Security ROI may be about 20% p.a. • So current expenditure maybe about right (but too little in small firms and too much in governments, big companies)

24. Privacy - Third Wave • Varian 96 - privacy as the right not to be annoyed by direct marketers - define rights better • When sending marketing pitches was expensive and evaluating them was cheap, we got too few messages and bought magazines. Now it’s the other way round and we buy spam filters • Huang 98 - regulation helps construct privacy preferences by steering people to one of many equilibria, which then stick

25. Privacy (cont’d) - Social Level • Odlyzko 2001 - pressure to price-discriminate is the main threat to privacy, and technology is making it steadily worse • End of bubble: privacy technology ventures had mostly failed - yet privacy costs billions, to business and consumers (Gellman 2002) • Taylor 2002: if data trading covert, firms gain more; otherwise high-value customers back off • Chellapa 2002: perceived security, privacy separate but correlated; it’s better for a firm to be trusted with privacy rather than just trusted

26. Privacy Themes - WEIS 2003 • Privacy paradox - most people say they value privacy, but act otherwise • May be due to myopic consumers (Syverson) • Lemons market for retailers (Vila, Greenstadt, Molnar) • Need a concrete solution to a clear threat (Shostack) • Shoppers care about privacy when buying clothes, but not cameras! Sensitivity focuses on items relating to personal image (Acquisti, Grossklags)

27. Privacy (cont’d) - social level • Varian / Wallenberg / Woloch, WEIS 2004 - privacy as ‘do not call’ strongly correlated with income - large study with DNC records • Mialon & Mialon 2004 - privacy as 4th amendment rights which cut intrusion directly but increase it indirectly (more crime). Technology lowers search costs -> society moves to exterior equilibrium of Swiss or Afghan type, depending on police accountability

28. Privacy - mechanism level • What sort of incentives will make people participate in remailer / P2P networks etc? • Acquisti / Dingledine / Syverson - free-rider problems in mix-nets, and options for clubs, reputation systems, preferential service etc • Danezis / Anderson - discretion is better • There’s now a whole workshop for P2P economics - many issues go across to privacy

29. Conclusions • Security and privacy spending seems to be determined in complex ways by assorted market failures • Firms, and governments, generally spend too much on security - they are risk-averse • Too little gets spent on privacy - consumers don’t care as much • To say much more, you have to be more specific about the type of security or privacy! Ultimately it’s all about power

30. More … • Economics and Security Resource Page – www.cl.cam.ac.uk/~rja14/econsec.html (or follow link from my home page • Economics of Privacy Page – www.heinz.cmu.edu/~acquisti/economics-privacy.htm