1 / 32

Robustness Testing: Discover unknown vulnerabilities with Testing & QA

Robustness Testing: Discover unknown vulnerabilities with Testing & QA. Ari Takanen Codenomicon Ltd. Be Proactive with Security. Modern security testing is about finding unknown zero-day vulnerabilities in devices and software before and after release

pams
Download Presentation

Robustness Testing: Discover unknown vulnerabilities with Testing & QA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Robustness Testing:Discover unknown vulnerabilities withTesting & QA Ari Takanen Codenomicon Ltd.

  2. Be Proactive with Security • Modern security testing is about finding unknown zero-day vulnerabilities in devices and software before and after release • Provides a quick technique for security assurance for any device or software • www.codenomicon.com/unknown/

  3. Security Vulnerability = Just A Bug

  4. Same Applies to (Legacy) Mobile Phones

  5. The Challenge

  6. Internet of Things = Future market for security and testing THINGS 50 B Digital Society Sustainable World Personal Mobile Inflectionpoints PEOPLE 5.0 B Global Connectivity ~0.5 B PLACES 1875 1900 1925 1950 1975 2000 2025 Source: Ericsson

  7. Codenomicon Labs Test Results http://www.codenomicon.com/labs/results

  8. Smart phone – attack surface WIRELESS: GPRS, EDGE/3G GSM, SMS, MMS, SMIL, OTA updates,... WIRELESS: 802.11: 802.11a/b/g/n, WPA, WPA2,.. WIRELESS: Bluetooth: L2CAP, RFCOMM, SDP, OPP, A2DP, AVRCP, PBAP, DUN,... IP CONNECTIVITY: IPv4 (ARP, ICMP, IGMP, IP, UDP, TCP), IPv6 (IP, ICMP, ND, RD, SEND, MLD, TCP, UDP), HTTP, TLS/SSL, OCSP, RTSP, SIP/IMS, RTP/RTCP, SigComp, DNS, MDNS, DHCP, NTP , SOAP, REST/JSON, SMTP, POP3, IMAP4, WAP/WMLC,.. PHYSICAL CONNECTIVITY: USB, SERIAL, MEMORY CARD, SIM,.. MEDIA: AUDIO (AAC, MP3, MP4, 3GP, WAV, ...), IMAGES (JPG, GIF, PNG, TIFF, ...), VIDEO (MPG1, MPG2, MP4/H.264, WEBM,... ), ARCHIVES (ZIP, JAR, CAB, ...), DOCUMENTS (PDF, DOC, PPT,..), X509, EMAIL (MIME, calendar, vcards,...), DRM, Flash, Java classes , Application installers,... [WEB] APPLICATIONS: XML, DRM, HTML5 (CSS, HTML, Javascript) , AT commands, inter process APIs/RPCs,

  9. Feature/conformance testing Performance/load testing Robustness testing Fuzzing Static Code Analysis Approaches to testing, how does fuzzing fit in?

  10. Microsoft SDL & fuzzing & static code analysis

  11. Microsoft SDL: Fuzz Here? Many organizations choose to deploy fuzzing in other parts of the SDL as well.

  12. Definition of fuzzing • Fuzzing is a technique for • intelligently and • automatically generating and passing into a target system • valid and • invalid message sequences to see if the system breaks, and if it does, what it is that makes it break.

  13. Product Security Terminology • Vulnerability – a weakness in software, a bug. • Threat/Attack – exploit against a specific vulnerability • Protocol Modeling – functional behavior, interface message sequences and message structures • Anomaly – abnormal or unexpected input • Failure – crash, busy-loop, memory corruption, or other indication of a bug in software

  14. Types of fuzzing • Random fuzzing • Apple 1980’s • Barton P. Miller 1980’s, 1990’s • Template based fuzzing • Capture traffic OR use sample files OR...  create mutated test cases • Specification based fuzzing • Model the specification, inject anomalies, transmit to target system

  15. Example Fuzzing Session

  16. What kinds of bugs does it find?

  17. Why We Must Fuzz? Mission Critical Device Life Cycle & Longevity Always On Never Off Update Frequency Try to secure devices that get infrequent updates or those needing very high severity updates out of band Designing systems for very long operational and legacy device support, security? “Always-on” applications or devices will have to deal with live updates, no down-time and still function in rugged/robust environments Mission critical devices will bring their own unique set of requirements – guaranteed up-time, high security and immunity from updates being an attack source

  18. Fuzzing vs. Common Criteria • Calculation of attack potential for Fuzzing tools:

  19. Attack Potential for Fuzzing Tools • Attack potential for fuzzing tools is 5-8… • What does that mean: • 0-9 = Basic = AVA_VAN.1-5 should not fail • 10+ = Enhanced Basic required at EAL4 • All Common criteria evaluated products should survive basic attacks such as fuzz-testing?

  20. Example: Traffic Capture Fuzzing

  21. “Models” and “Rules”

  22. Scaling Fuzz Tests Robustness is also about performance, and therefore model-based tools have to be fast in generating test cases

  23. Testing In The Cloud Test the interior and exterior of the cloud, including services, devices, applications, and hypervisor stability

  24. ”[FUZZING] tools are *amazing*. Using them is like being attacked by the most relentless adversary who uses every possible method to find flaws in your codeWe fixed subtle crash bugs in Samba that had been in the code for over ten years. We would *never* have found those bugs without the [FUZZING] tools. If you're serious about implementing protocols correctly, you need [FUZZING] tools.“-- Jeremy Allison, Co Creator of Samba. Model-based Fuzz-Testing Examples

  25. Model-based Fuzz-Testing Examples

  26. Model-based Fuzz-Testing Examples

  27. Model-based Fuzz-Testing Examples

  28. Model-based Fuzz-Testing Examples

  29. Model-based Fuzz-Testing Examples

  30. Conclusions • Why is fuzzing always an excellent choice for a testing solution... • ... and sometimes the only feasible one? • Easy to automate, systematic, top coverage, top efficiency • Increasingly widely adopted, some contractors/customers require it • Real life examples indicate: you will find security critical bugs by fuzzing

  31. THANK YOU – QUESTIONS? PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS “Thrill to the excitement of the chase! Stalk bugs with care, methodology, and reason. Build traps for them. ....Testers!Break that software (as you must) anddrive it to the ultimate- but don’t enjoy the programmer’s pain.” [from Boris Beizer]

More Related