Anti-Phishing Phil The Design and Evaluation of a Game That Teaches People Not to Fall for Phish

1. Anti-Phishing PhilThe Design and Evaluation of a Game That Teaches People Not to Fall for Phish S. Sheng, B. Maginien, P. Kumaraguru, A. Acquisti, L. Cranor, J. Hong, E. Nunge

3. Phishing email Subject: eBay: Urgent Notification From Billing Department

4. We regret to inform you that you eBay account could be suspended if you don’t update your account information.

5. https://signin.ebay.com/ws/eBayISAPI.dll?SignIn&sid=verify&co_partnerid=2&sidteid=0https://signin.ebay.com/ws/eBayISAPI.dll?SignIn&sid=verify&co_partnerid=2&sidteid=0

7. What is phishing? • Social engineering attack • Misrepresents electronic identity • Tricks individuals into revealing personal credentials • Defrauds users Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service industry perspective. 2005.

8. Countermeasures for phishing • Silently eliminating the threat • Regulatory & policy solutions • Email filtering (SpamAssasin) • Warning users about the threat • Toolbars (SpoofGuard, TrustBar) • Training users not to fall for attacks

9. Design Rationale • Security is a secondary task • Learning by doing • Fun and engaging • Better strategies

10. Anti-Phishing Phil • Online game • http://cups.cs.cmu.edu/antiphishing_phil/ • Teaches people how to protect themselves from phishing attacks • Identify phishing URLs • Use web browser cues • Find legitimate sites with search engines

17. More about the game • Four rounds • Two minutes in each round • Increasing difficulty • Eight URL “worms” in each round • Four phishing and four legitimate URLs • Users must correctly identify 6 out of 8 URLs to advance • In-between round tutorials

18. User Study • Test participants’ ability to identify phishing web sites before and after training • 10 URLs before training, 10 after, randomized • Up to 15 minutes of training • Training conditions: • Web-based phishing education • Tutorial • Game • 14 participants in each condition • Screened out security experts • Younger, college students

21. Results • No significant difference in false negatives among the three groups • Game group had fewest false positives

22. The effects • Improvement could be due to • Learning to distinguish legitimate from phish • Raising suspicion about all web sites • Learning is better than raising suspicion • Fewer false positives • Will help people more in the long run

23. Conclusions • Used signal detection theory to measure effects • Existing training materials increased suspicion with little learning • Game did not raise suspicion but resulted in players learning to distinguish legitimate from phish • In some cases a little more suspicion would have helped • Game condition performed best overall!

24. Acknowledgements • Members of Supporting Trust Decision research group • Members of CUPS lab

25. Play Anti-Phishing Phil: http://cups.cs.cmu.edu/antiphishing_phil/ CMUUsablePrivacy andSecurityLaboratoryhttp://cups.cs.cmu.edu/

26. Falling for Phishing

27. Misidentifying Legitimate Sites

28. Lessons Learned • Pilot test • Users be able to identify phishing • But they misidentify real ones • Users tend to get the specifics, but not the underlying concepts • Conceptual – procedural knowledge • User didn’t ask father for help too much