Show Your Vulnerable Side: How to do a Vulnerability Assessment

1. Talk for the 50th Annual ASIS Conference, Sept 26-30, 2004 (Dallas, TX) Show Your Vulnerable Side:How to do a Vulnerability Assessment Roger G. Johnston, Ph.D., CPP Vulnerability Assessment Team Los Alamos National Laboratory 505-667-7414 rogerj@lanl.gov http://pearl1.lanl.gov/seals.default.htm LAUR-04-4147

2. LANL Vulnerability Assessment Team Physical Security • consulting • cargo security • tamper detection • nuclear safeguards • training & curricula • vulnerability assessments • novel security approaches • new tags & seals (patents) • unique vuln. assessment lab The VAT has done detailed vulnerability assessments on hundreds of different security devices, systems, & programs. The greatest of faults, I should say, is to be conscious of none. -- Thomas Carlyle (1795-1881)

3. Physical Security This talk will focus primarily on vulnerability assessments of physicalsecurity, but presumably many of the ideas and principles also apply to other types of security such as: • computer security • network & Internet security • intellectual property security • information & records security • communications security Better be despised for too anxious apprehensions, than ruined by too confident security. -- Edmund Burke (1729-1797)

4. Definitions physical security: trying to protect valuable, tangible assets from harm. Examples of assets needing protection: Security Guard: “Don't make me take off my sunglasses!” -- From the movie Bringing Out the Dead (1999)

5. Definitions (con’t) The “harm” that we wish to avoid might involve: The ultimate security is your understanding of reality. -- H. Stanley Judd

6. Definitions (con’t) VAs vulnerability assessment (VA): discovering and demonstrating ways to defeat a security device, system, or program. Should include suggesting counter-measures and security improvements. He that wrestles with us strengthens our skill. Our antagonist is our helper. -- Edmund Burke (1729-1797)

7. Physical Security is Difficult! Before thinking about how to assess physical security, we need to recognize that it is difficult and there are no guarantees of success. Especially because complacency, over-confidence, wishful thinking, and arrogance are not compatible with good security. Danger breeds best on too much confidence. -- Pierre Corneille (1606-1684)

8. The traditional performance measure for security is pathological: success is often defined as nothing happening. Cost/Benefit analysis is difficult. There are few meaningful standards, fundamental principles, models, or theories. Everything is a compromise & a tradeoff. Why Physical Security is So Difficult There is always more spirit in attack than in defense. -- Titus Livius (59 BC)

9. Objectives are often remarkably vague. Security managers & personnel aren’t always creative or proactive, but adversaries may be. Adversaries and their resources are usually unknown to security managers, yet the adversaries understand the security systems. Society & employees often do not like security. Why Physical Security is So Difficult (con’t) We spend all our time searching for security, and then we hate it when we get it -- John Steinbeck (1902-1968)

10. Effective security management is highly multi-disciplinary: engineering, computer science, psychology, sociology, management, economics, communication, & law. Adversaries can attack at one point, but security managers may need to protect extended assets. Adversaries need exploit only one or a small number of vulnerabilities, but security mangers must identify, prioritize, & manage many vulnerabilities, including unknown ones. Why Physical Security is So Difficult (con’t) We have to get it right every day and the terrorists only have to get it right once. So we have to be ahead of the game. --TSA Spokeswoman Lauren Stover

11. Security functions are often tedious. Security personnel have trouble identifying security vulnerabilities because they don’t want them to exist. (It’s hard to think like the bad guys if you devote your career to being a good guy.) Why Physical Security is So Difficult (con’t) No problem can be solved from the same consciousness that created it. -- Albert Einstein (1879-1955)

12. Physical Security scarcely a “field” at all! - You can’t (for the most part) get a degree in it. - Not widely attracting young people, females, the best and the brightest. - Few peer-review, scholarly journals or R&D conferences. - Lots of snake oil salesmen. - Shortage of models, fundamental principles, metrics, rigor, standards, guidelines, critical thinking, & creativity. - Overly macho and often dominated by bureaucrats, committees, groupthink, “old boys” networks, linear/concrete/wishful thinkers. Why Physical Security is So Difficult (con’t) The only security is the constant practice of critical thinking. -- William Graham Sumner (1840-1910)

13. Security Survey Risk Management (“Design Basis Threat”) Vulnerability Assessment Major Tools for Improving Security If we don't succeed, we run the risk of failure. -- Dan Quayle

14. Not really the same thing because they produce different results. The task of identifying Threats & Vulnerabilities, done as part of Risk Management (or DBT), is too often not really a Vulnerability Assessment. Security Surveys and Risk Management/DBT were major breakthroughs & are still useful… But they are not enough! Security Surveys vs. Risk Management vs. VAs Men do not like to admit to even momentary imperfection. My husband forgot the code to turn off the alarm. When the police came, he wouldn't admit he'd forgotten he code... he turned himself in. --Rita Rudner

15. Basically a management walk around. Walk the spaces, looking for security problems. A checklist is often used. Security Survey We made too many wrong mistakes. -- Yogi Berra

16. Binary Close-ended Often unimaginative Not focused on adversaries Overly focused on the check list Does not encourage new countermeasures Expectation that problems will leap out at you Limitations of Security Surveys 0 1 It's better to be looked over than overlooked. -- Mae West, Belle of the Nineties, 1934

17. Similar to Risk Management Techniques in other fields. Identify Assets, Threats & Vulnerabilities, Adversaries, Consequences, Safeguards & Countermeasures. Assign relative priorities and probabilities. (Generate lots of tables.) Field your resources appropriately. Risk Management The first step in the risk management process is to acknowledge the reality of risk. Denial is a common tactic that substitutes deliberate ignorance for thoughtful planning. -- Charles Tremper

18. “Design Basis Threat” is similar to Risk Management. DBT basically means “design your security to deal with the current real-world threats”. In practice, DBT tends to focus more on hardware and infrastructure than Risk Management does. Design Basis Threat (DBT) A hypothetical paradox: what would happen in a battle between an Enterprise security team, who always get killed soon after appearing, and a squad of Imperial Stormtroopers, who can't hit the broad side of a planet? -- Tom Galloway

19. There is rarely any guidance on how to determine the Threats & Vulnerabilities other than looking at past security incidents. But that is being reactive, not proactive. Not good enough post-9/11, in a rapidly changing world, or for dealing with rare catastrophic events. Still binary & close-ended Limitations of Conventional Risk Management (or DBT) You can never plan the future by the past. -- Edmund Burke (1729-1797)

20. Often done unimaginatively The attack probabilities are usually a fantasy Suffers from overconfidence in tables and the “fallacy of precision” Not done from the perspective of the adversaries More Limitations of ConventionalRisk Management (or DBT) The time to repair the roof is when the sun is shining. -- John F. Kennedy (1917-1963) 3.14159265359

21. Tendency to let the good guys and existing security measures define the adversaries & attack modes Often used to justify the status quo--typically does not encourage new countermeasures Ignores simple/cheap countermeasures when the attack probabilities are judged (rightly or wrongly) to be low or zero More Limitations of ConventionalRisk Management (or DBT) It isn't that they can't see the solution. It is that they can't see the problem. -- G.K. Chesterton, The Scandal of Father Brown (1935)

22. Perform a mental coordinate transformation and pretend to be the bad guys. (This is a lot harder to do than one might think.) Gleefully look for trouble, rather than seeking to reassure yourself that everything is fine. Unlike Security Surveys or Risk Management, don’t let the good guys define the problem or its parameters. Vulnerability Assessment It is sometimes expedient to forget who we are. -- Publilius Syrus (~42 BC)

23. Example: Open Window security survey:issue orders to close & lock window! risk management:ignore if not envisioned as part of a specific threat or attack from a likely adversary; otherwise, design procedure to close & lock window. VA:Oh boy, an open window! What mischief can this lead to? You can observe a lot by just watching. -- Yogi Berra

24. Fully understand the device, system, or program and how it is REALLY used. Talk to the low-level users. Play with it. Brainstorm--anything goes! Play with it some more. Vulnerability Assessment Steps Scientists are the easiest to fool. They think in straight, predictable, directable, and therefore misdirectable, lines. The only world they know is the one where everything has a logical explanation and things are what they appear to be. Children and conjurors--they terrify me. Scientists are no problem; against them I feel quite confident. -- Spoken by Zambendorf in Code of the Lifemaker, (James Hogan, 1987)

25. Edit & prioritize potential attacks. Partially develop some attacks. Determine feasibility of the attacks. Devise countermeasures. Vulnerability Assessment Steps It's awful hard to get people interested in corruption unless they can get some of it. -- Will Rogers (1879-1935)

26. Perfect attacks. Demonstrate attacks. Rigorously test attacks. Rigorously test countermeasures. Vulnerability Assessment Steps A thing may look specious in theory, and yet be ruinous in practice; a thing may look evil in theory, and yet be in practice excellent. -- Edmund Burke (1729-1797)

27. Brain Storming Nothing can inhibit and stifle the creative process more--and on this there is unanimous agreement among all creative individuals and investigators of creativity--than critical judgment applied to the emerging idea at the beginning stages of the creative process. ... More ideas have been prematurely rejected by a stringent evaluative attitude than would be warranted by any inherent weakness or absurdity in them. The longer one can linger with the idea with judgment held in abeyance, the better the chances all its details and ramifications [can emerge]. -- Eugene Raudsepp, Managing Creative Scientists and Engineers (1963). In theory there is no difference between theory and practice. In practice there is. -- Yogi Berra

28. What if you can’t have or afford outside vulnerability assessors? Use smart, hands-on, creative people inside your organization who are not associated with security. Seek: wise guys, trouble makers, smart alecks, schemers, organizational critics, loophole finders, questioners of tradition and authority, outside-the-box thinkers, artists, hackers, tinkerers, problem solvers, & techno-nerds. Could Hamlet have been written by committee, or the Mona Lisa painted by a club? Could the New Testament have been composed as a conference report? Creative ideas don't spring from groups. They spring from individuals. -- Alfred Whitney Griswold (1885-1959)

29. Vulnerabilities are often obvious to outsiders… To see what is in front of one's nose needs a constant struggle. -- George Orwell (1903-1950)

30. Other Reasons for Doing a Vulnerability Assessment • mental rehearsal • fresh perspectives • fun/relieves tedium • increased alertness • bluffing (don’t underestimate) • enhanced sense of professionalism • educational/professional development for security staff • can involve other members of the organization, thus increasing employees’ security awareness • can help justify additional resources for security Without deviation from the norm, progress is not possible. -- Frank Zappa (1940-1993)

31. No meaningful standards or underlying theory Defeats are a matter of degree & probability No clear endpoint Wishful thinking is hard to avoid. Tricky Aspects of Vulnerability Assessments (VAs) Nothing is easier than self-deceit. For what each man wishes, that he also believes to be true. -- Demosthenes (382-322 BC)

32. Recursion (chasing a moving target) Most security failures are due to human error, which is hard to model and predict. Testing/Demonstration realism can be difficult to achieve. Tricky Aspects of VAs (con’t) We are never deceived; we deceive ourselves. -- Johann Wolfgang von Goethe (1749-1832)

33. No conflicts of interest or wishful thinking. No “Shoot the Messenger” Syndrome. No retaliation or punishment against security personnel or managers when vulnerabilities are found. Use of independent, imaginative assessors who are psychologically predisposed to finding problems and suggesting solutions, and who (ideally) have a history of doing so. General Attributes of Effective VAs When people are engaged in something they are not proud of, they do not welcome witnesses. In fact, they come to believe the witness causes the trouble. -- John Steinbeck (1902-1968)

34. No binary view of security. Rejection of a finding of zero vulnerabilities. Rejection of the idea of “passing” the VA, or of VAs as “certification”. Discovering vulnerabilities is viewed as good (not bad) news. Attributes of Effective VAs When we were children, we used to think that when we were grown-up we would no longer be vulnerable. But to grow up is to accept vulnerability... To be alive is to be vulnerable. -- Madeleine L'Engle

35. Done early, iteratively, and periodically . Done holistically, not by component, sub-system, function, or layer. (Attacks often occur at interfaces.) No unrealistic time or budget constraints on the VA, or on what attacks or adversaries can be considered. Done in context. Attributes of Effective VAs He that will not apply new remedies must expect new evils; for time is the greatest innovator. -- Francis Bacon (1561-1626)

36. No underestimation of the cleverness, knowledge, skills, dedication, or resources of adversaries. The good guys don’t get to define the problem, the bad guys do. Simple, low-tech attacks are examined first. Attributes of Effective VAs A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools. -- Douglas Adams (1952-2001)

37. Findings are reported to the highest appropriate level without editing, interpretation, or censorship by middle managers. No confusion about the difference between VAs and other kinds of hardware testing (materials, environ-mental, ergonomic, field readiness) or personnel testing. Attributes of Effective VAs The first principle is that you must not fool yourself-- and you are the easiest person to fool. -- Richard Feynman (1918-1988)

38. The following attacks are all considered: • fault analysis • false alarming • poke the system • wait & pounce • backdoor attacks • impersonation • social engineering • tampering with security training • insiders, outsiders, insiders + outsiders Attributes of Effective VAs Evil is easy, and has infinite forms. -- Blaise Pascal (1623-1662)

39. Rohrbach’s Maxim must be considered: No security system will ever be used properly (the way it was designed) all the time. Shannon’s Maxim must be considered: The adversaries know and understand the security systems, strategies, and hardware being used. Attributes of Effective VAs Inanimate objects can be classified scientifically into three major categories; those that don't work, those that break down and those that get lost. -- Russell Baker Everything secret degenerates … nothing is safe that does not show how it can bear discussion and publicity. -- attributed to Lord Action (1834-1902)

40. The vulnerability assessors need to praise the good things because: + We want the good things to be recognized and to continue. + Security managers need to be willing to arrange for future VAs. + Discussing the good things will make security managers more willing to hear about potential problems. It should be clear up front that the vulnerability assessment will produce more suggestions and countermeasures than are likely to be implemented. Security mangers (not the assessors) should ultimately decide which (if any) make sense to employ. Attributes of Effective VAs Our only security is our ability to change. -- John Lilly

41. Don’t Overlook the Insider Threat! • The insider threat is often overlooked or underestimated, and can be very difficult to deal with. • Disgruntled employees are a particular insider threat. We have met the enemy and he is us. -- Walt Kelly, the words of Pogo in Earth Day 1971 cartoon strip

42. Disgruntled Workers • Research shows that employee disgruntlement is associated with perceptions of unfairness & inequity, not necessarily objective conditions. • Disgruntled employees are known to be a risk for workplace violence, espionage, theft, & sabotage. What has posterity ever done for me? -- Groucho Marx (1890-1977) Honesty may be the best policy, but it's important to remember that apparently, by elimination, dishonesty is the second-best policy. -- George Carlin

43. Workplace Violence (USA) • ~ 1 million victims of workplace violence each year • >1000 workers killed each year due to workplace homicide • Homicide is the number one cause of on-the-job deaths for female employees Source: NIOSH Always go to other people’s funerals. Otherwise they might not come to yours. --Yogi Berra

44. Causes of Increasing Worldwide Employee Disgruntlement • global downsizing & outsourcing • weakening of labor unions & collective bargaining • increased use of temp & limited-term employees • the disappearance of lifetime employment • increased workforce diversity We have to distrust each other. It's our only defense against betrayal. -- Tennessee Williams (1911-1983)

45. Causes of Increasing World-Wide Employee Disgruntlement(con’t) • technical obsolescence • the rapid pace of organizational change • increased whistle-blowing • depersonalization caused by increased urbanization, expanding bureaucracy, the growth of multinational corporations, and the increased use of email & virtual meetings No one can build his security upon the nobleness of another person. -- Willa Cather (1873-1947)

46. Disgruntled Americans • American employees are particularly at risk • for disgruntlement due to characteristic traits: • identity is based on work • work long hours • strong individualism • traditional belief in fairness • traditional belief in “American Dream” Americans do not abide very quietly the evils of life. -- Richard Hofstadter In every American there is an air of incorrigible innocence, which seems to conceal a diabolical cunning. -- A. E. Housman (1859-1936)

47. Disgruntlement Countermeasures • Listen, acknowledge, validate, & empathize with employees. • Allow employees to freely offer suggestions & concerns. • Have legitimate complaint resolution processes. Too often these are non-existent, ineffective, adversarial, or fraudulent, especially in large or bureaucratic organizations. This is very dangerous (and bad for productivity). • Be aware that employee perceptions about fairness are the only reality. • Treat departing employees & retirees well. Sincerity is everything. If you can fake that, you've got it made. -- Comedian George Burns (1896-1996)

48. Also, Don’t Forget About… Computer & Computer Media physical security! Relations with public, neighbors, & local authorities Effective security awareness training for all employees Even if you're on the right track, you'll get run over if you just sit there. -- Will Rogers (1879-1935)

49. Or about having plans to deal with… Espionage Sabotage Terrorism Natural Disasters War & Civil Unrest Product Tampering Illness & Epidemics Industrial Accidents Strikes & Labor Unrest When choosing between two evils, I always pick the one I never tried before. -- Mae West (1893-1980)

50. Product Tampering Tamper-Evident Packaging Model of how to effectively deal with product tampering: J&J On a bag of Fritos: You could be a winner! No purchase necessary. Details inside.