Web App Security – The Good, the Bad and the Ugly

1. Web App Security – The Good, the Bad and the Ugly Ross Anderson Cambridge University

2. Is Web 2.0 Reinventing the Whole World?

3. So what’s changed? • A cynic might say that IT just goes in cycles! • Back in the 60s and 70s, we had mainframe bureau services • Then we had minis, then PCs • The pendulum seems to be swinging back – server farms do what mainframes used to • And we get a wide range of terminals – phones, netbooks, PCs, … • How should we make sense of all this?

4. Economics and Security • About 2000, we realised that engineering analysis alone didn’t explain all that goes wrong • Economic analysis often explains failure better! • Electronic banking: UK banks were less liable for fraud, so became careless and ended up suffering more internal fraud and errors • Distributed denial of service: viruses now don’t attack the infected machine so much as use it to attack others • Why is Microsoft software so insecure, despite market dominance?

5. New View of Infosec • Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives • Medical record systems bought by research or finance directors, not patients – so failed to protect privacy • Casino websites suffer when infected PCs run DDoS attacks on them • Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution

6. IT Economics (1) • The first distinguishing characteristic of many IT product and service markets is network effects • Metcalfe’s law – the value of a network is the square of the number of users • Real networks – phones, fax, email • Virtual networks – PC architecture versus MAC, or Symbian versus WinCE • Network effects tend to lead to dominant-firm markets where the winner takes all

7. IT Economics (2) • Second common feature of IT product and service markets is high fixed costs and low marginal costs • Competition can drive down prices to marginal cost of production • This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … • These effects can also lead to dominant-firm market structures

8. IT Economics (3) • Third common feature of IT markets is that switching from one product or service to another is expensive • E.g. switching from Windows to Linux means retraining staff, rewriting apps • Shapiro-Varian theorem: the net present value of a software company is the total switching costs • So major effort goes into managing switching costs – once you have $3000 worth of songs on a $300 iPod, you’re locked into iPods

9. IT Economics and Security • High fixed/low marginal costs, network effects and switching costs all tend to lead to dominant-firm markets with big first-mover advantage • So time-to-market is critical • Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ was quite rational • Whichever company had won in the PC OS business would have done the same • “Growth is primary, revenue is secondary” – Mark Zuckerberg

10. IT Economics and Security (2) • When building a network monopoly, you must appeal to vendors of complementary products • That’s application software developers in the case of PC versus Apple, then of Symbian versus Windows/Palm, now Facebook • Lack of security in early Windows / Symbian / Facebook made life easier for them • So did the choice of security technologies that dump costs on the user (SSL, not SET) • Once you’ve a monopoly, lock it all down!

11. Security Economics and Web Applications • The big security economics problem is aligning incentives • The big system engineering problem is managing complexity. You want architecture, i.e. interfaces, to divide up systems sensibly • Consider a travel agent, buying services from airlines, hotels etc. It pretty much all lines up • Open interfaces, defined by contract • Competition drives costs down, usability up

12. Security Economics and Web Applications (2) • However, some web apps are platforms, so operate under the same forces as Windows or Symbian or S/360 • E.g. Facebook – huge network effects • Incentives on its developers: • grab the market now, fix privacy later • appeal to complementers (app writers) • But does social context change anything?

13. How Fraud Adapts to SNS • The old scams are still there – 419, spam, phishing, XSS, malware, click fraud, … • Social context makes phishing more effective (72% in controlled study – Jagatic) not to mention targeted attacks / scams • Facebook now 7th biggest phishing target (after PayPal, top banks, eBay) • Frequent genuine emails with login links • Some incentive on operator to fight it (spam caused decline of MySpace, Friendster)

14. Privacy • Most people say they value privacy, but act otherwise. Most privacy ventures failed. Why? • Odlyzko – technology makes price discrimination both easier and more attractive • Acquisti – people care about privacy when buying clothes, but not cameras • Loewenstein – privacy is heavily context sensitive. People only really worry if salient • Facebook viruses ‘worse’ than PC viruses (as more personal) or not (as less salient)?

15. Privacy and SNS • Conflict of interest • Facebook wants to sell user data • Users want feeling of intimacy, small group, social control • Very complex access controls – over 60 settings on 7 pages • Over 90% of users never change defaults • The complexity lets Facebook blame the customer when things go wrong

16. Privacy and SNS (2)

17. Privacy and SNS (3) • See our paper ‘Eight friends are enough’ • Given the eight published friends, an outsider can run all the usual network analysis • Including covert community detection as used by the spooks

18. Security Economics and Web Applications (3) • As you’d expect from the incentives, Facebook provides the appearance of security, not reality – ‘security theatre’ • Abd it deals with the occasional outrage using ‘democracy theatre’ (see our blog, www.lightbluetouchpaper.org for more) • Is this sustainable? • Long-term problem: European regulators

19. Security Economics and Web Applications (4) • Sometimes the monopoly doesn’t come from platform dynamics but exogenously • Example: UK attempt to centralize all medical records, children’s records • Records at GPs, hospitals being moved to ‘hosted’ systems • Sales pitch: benefits of research • Driver: bureaucratic centralization • Gotcha: I v Finland

20. Security Economics and Web Applications (5) • Thankfully the UK TG programme is failing; see our report “Database State” for more • But might Google or Microsoft make a health-record web service work? • There are similar incentives on private and public sectors to collect data in order to price discriminate between clients / citizens • Are there any technical limits (systems complexity, microeconomics) or must we rely on our legislators and courts?

21. The Gladman Principle “You can have security, or functionality, or scale. With good engineering you can have any two of these. But there’s no way you can get all three.” Brian Gladman (formerly of UK Defence Science Advisory Board)

22. Compartmentation • It’s OK to have 20 doctors and nurses having access to 10,000 patients’ records in a medical practice • With some care, it’s just about OK to have 2000 doctors and nurses having access to 1,000,000 patients’ records in a hospital • It’s not OK to have 580,000 health service staff having access to 50,000,000 citizens’ records on a national database • … as our Prime Minister has learned …

23. Attack Trends • One aspect of security economics is building models that explain how things go wrong • Another is the econometrics – measuring what actually does go wrong • We have a research project on collecting statistics on spam, phishing, malware (see my Google tech talk, for example) • Recent trends in malware are getting worrying! • If an attack can be industrialized, it will be …

24. Case study – the Dalai Lama • Simple attacks reported on the Office of His Holiness the Dalai Lama (OHHDL) since 2007 • From directed spam to simple targeted attacks • Compromise became obvious in July 2008 – foreign diplomats about to meet the Dalai Lama were warned off • We got asked to investigate

25. Modus Operandi • A sends email to B on topic X, archived publicly • C sends email to A pretending to be B, on topic X, with toxic attachment • C pretending to be A takes over mail server • Internal mail attachments thereafter toxic • PCs then accessed remotely … • We call this ‘Social Malware’ • The typical company has no defence at all!

26. A low grade sample

27. Malware Equilibrium? • Big change in 2004: black market led to specialisation • Malware now professionally written; most exploits are for money, not bragging rights • Most companies just don’t know how to block social malware (even Deloittes was among the victims of the Chinese) • What will the world be like if 1%, or 5%, or machines are 0wned, and exploited?

28. Open versus Closed? • Are open systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them • This debate goes back to the 17th century! • Theorem (2002): openness helps both equally if bugs are random and standard dependability model assumptions apply • So whether open is better than closed will depend on whether / how your system differs from the ideal

29. The Good, the Bad and the Ugly • Travel agent: not a big deal if the bad guys occasionally go on holiday (the bank pays) • Facebook: there will be all sorts of platform exploits, and social exploits, with which they’ll have to cope. As for compromised user machines, my daughter’s view … • Government databases: you can’t make everyone’s medical records available to 500,000 doctors and nurses and still have privacy • The insider (malware) threat sets limits here!

30. An Opportunity • If 1% of end-user machines will always be infected with malware, what can we do? • Web services can offer a haven • But they need to assume some corrupt insiders • Experience from defence – compartmentation • And from accounting – dual control, audit, backup, … • How do you build these ideas into other apps? • What other limits on security, functionality and scale are there – and what’s the social angle?

31. The Research Agenda • The online world and the physical world are merging – many years of turbulence ahead! • If Web 2.0 is going to reinvent the world, expect it to reinvent the problems too • The security world is changing, though • The old paradigm was what might go wrong … • Security economics gives us tools to think about what people might want things to go wrong, and metrics to measure what’s actually going wrong

32. More … • See www.ross-anderson.com for survey articles, our ENISA and Tibet reports, and my security economics resource page • WEIS – Workshop on Economics and Information Security – UCL, June 24–5 • Workshop on Security and Human Behaviour – in Cambridge in 2010 • ‘Security Engineering – A Guide to Building Dependable Distributed Systems’