Avoiding Pitfalls of Static Analysis

1. Avoiding Pitfalls of Static Analysis Abraham Kang ASD-W02 Senior Director Consultant @KangAbraham

2. Special Thanks Sung Min Hong (Staff Engineer on my team) RomainGaucher (Security Researcher at Coverity) Alvaro Muñoz (Principle Security Researcher at HP Fortify) Amit Ashbel (Product Marketing at Checkmarx) OhadRabinovich (Engine Product Manager at Checkmarx)

3. Interaction Is Important This is not your normal one way presentation I am looking to get feedback from you so speak up and ask questions

4. About Me Worked with a static analysis tool as an end user at Wells Fargo for over 4 years running scans, reviewing results, and assessing risk on web, infrastructure and desktop applications Worked as Principle Security Researcher at HP Fortify (static analysis tool vendor) writing custom rules and doing research on security vulnerabilities to improve Fortify scan results Working as Senior Director of security team building security solutions and successfully applying static analysis tools to mobile, infrastructure, and payment solutions

5. Who Are You • Backgrounds • QA Testers • Security Testers • Managers • How many are doing security testing? • Static Analysis • Dynamic Analysis • Reversing

6. Can You Relate? • Typical static analysis buying scenario • How much training did you get on the tool? • How much of that covered writing custom rules for your tool?

7. Overall Process Assess results Add custom rules Assess results Scanning & Verification Web Application, Library, Trusted App, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis Understand What you are Scanning Verify the Integrity of Your Scan Add Custom Rules Add Canaries Understand What Static Analysis Cannot Do

8. What are You Scanning? Assess results Add custom rules Assess results Scanning and Verification Web Application, Trusted App, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis • Well Supported • Web Application • Using popular frameworks Struts, Spring MVC,etc. • Languages • Java, C/C++, Ruby • Not as well supported (your mileage may vary) • Android Applications • Smart Card Apps (More Esoteric) • Trusted Execution Environment (Extremely Esoteric) • Operating System Components (Kernel, Drivers, etc.) • Mixed Languages (Java to C/C++ and vice versa) • Scripting Language (Javascript, PHP, Python, Ruby) • Niche Languages (Closure, Erlang, Haskell, Scala, etc.) • Difficult to Scan because of Dependencies • Libraries • Why is it like this?

9. Integrity of Static Analysis Scans Assess results Add custom rules Assess results Scanning and Verification Web Application, Trusted App, Library, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis • The type of application and framework you are scanning? • Is is well supported by the existing rules of the tool • Did all of your source and application artifacts get scanned by your static analysis tool? • Where there any errors during the translation? • Where there any errors during the analysis phase? • All these worked fine…Now What?

10. Important Background Information Intermediate Representation Control Flow, Dataflow, etc. Analysis Source Code You need to understand how static analysis tools work generally Source files are translated into intermedia representations for eventual analysis as files /src/directory/Orginal.java=> /static-analysis/engine/build_id/directory/Original.java.astor In Coverity you can see which files were translated by looking for “cov-translate” in your build.log or looking in the cov-build –dir<idir> directory In Checkmarx<EngineInstall Dir>\Logs\ScanLogs\<Project name>\YYYYMMDDT######-##_#_#########_############_#-##########_####.log What do you want to do to verify that this stage of the process worked completely?

11. Understanding How Your Tool Works Intermediate Representation Control Flow Graph, Data Flow Graph, etc. Analysis Source Code You need to understand how static analysis tools work generally Make sure that all of your source files have been translated

12. Common Problems During Translation • Modified build scripts have errors in them • The static analysis tool is assuming the wrong version of the language parsed • Java 1.5 and 1.8 have significant changes • Make sure that your language compiler is recognized by the static analysis tool • Dependent libraries are not included • Setting $LD_PRELOAD • Using ”chroot” or “fakeroot” • Statically linking to OS • Forking (-Dmaven.compiler.fork=true) or sharing compilation (UseSharedCompilation=true) • Most of these issues can be resolved by addressing the issues in the debug log or adding the missing jar files to the classpath or *.so/*.dll to the appropriate path • Change build scripts to avoid problematic constructs • Some issues will not be resolved due to miss match in the parsers utilized by the tools being different from your build environment • Checkmarx does not have a dependency to the build representation of the code but may suffer some degradation in the fidelity of the representation.

13. Common Problems During Analysis Identified by looking at the logs (analysis.log, sca.log, etc.) for strings like: “analysis failed on ”, “error”, “exception”, etc. Can be resolved by:Calling support for your productIdentifying starved system resources (usually disk or memory)

14. Looks like the Translation and Analysis are Doing Well (Now What?) Assess results Add custom rules Assess results Scanning & Verification Web Application, Trusted App, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis Deeper understanding Custom rules Canaries

15. Custom Rules Applied to Vulnerable Code Flows Passthrough3rd Party Library Sink Source Sources and Entrypoints Passthroughs Sinks

16. Sources and Entrypoints Passthrough3rd Party Library Sink Source • Sources and Entrypoints • Where untrusted (possibly attacker controlled) input enters the system • Sources can be identified by runtime functions calls in the application • String taintedData1= obj.funct(taintedData2); • Entrypoints are slightly different as untrusted data is bound into the parameters of function definitionsby the system or framework • Resolution funct(String taintedData1, String taintedData2){ • When do you need to create Sources or Entrypoint Rules? • Are you using a less popular framework which provides user input to the application?

17. Passthroughs Passthrough3rd Party Library Sink Source • Passthroughs • Required by some static analysis tools for taint propagation in functions that the static analysis tool cannot understand due to missing source code or use of a 3rd party library • String continuedTaint= java.lang.String.format (“SELECT * FROM … %s”, taintedData1) • When do you need to create passthrough rules? • Is untrusted data processed by 3rd party libraries before being passed onto backend systems

18. Sinks Passthrough3rd Party Library Sink Source • Sinks • Locations (usually parameters of function calls) where if untrusted data were to reach, the attacker would be able to cause a vulnerability • connection.execute(continuedTaint) • When do you need to create Sink rules? • Is untrusted data processed by 3rd party libraries before being passed onto backend systems

19. Dataflow Source Rules A function that returns possibly attacker controlled data via its in-out type parameters or return value. Checkmarx Return All.FindByMemberAccess(“ServletRequest.getParameter”)  Fortify<DataflowSourceRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID><FunctionIdentifier> <NamespaceName><Pattern>javax\.servlet</Pattern></NamespaceName> <ClassName><Pattern>ServletRequest</Pattern></ClassName> <FunctionName><Pattern>getParameter</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> <OutArguments>return</OutArguments> <TaintFlags>+WEB,+XSS</TaintFlags> </DataflowSourceRule>

20. Dataflow Source Rules (con’t) A function that returns possibly attacker controlled data via its in-out type parameters or return value. Coverity* {  "taint_kind": "servlet",  "tainted_data": {    "return_value_of": {      "or": [        {          // The native getParameter call          "matching": "^javax\\.servlet\\.ServletRequest\\.getParameter\\("        },        {          // All implementations that override the getParameter          "overrides": "^javax\\.servlet\\.ServletRequest\\.getParameter\\("        }      ]    }  }}

21. Dataflow Entrypoint Rules A rule used to taint parameters going into the method Checkmarx All.GetParameters(All.FindByShortName("main").FindByType(typeof(MethodDecl)),0) Fortify<DataflowEntryPointRule language="java" formatVersion="3.8"> <RuleID>xxxx</RuleID> <FunctionIdentifier> <NamespaceName> <Pattern>.*</Pattern></NamespaceName> <ClassName><Pattern>.*</Pattern></ClassName> <FunctionName><Pattern>main</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"> <Parameters><ParamType>java.lang.String[]</ParamType></Parameters> <Modifiers><Modifier>static</Modifier></Modifiers></FunctionIdentifier> <InArguments>0</InArguments> <TaintFlags>+WEB,+XSS</TaintFlags> </DataflowEntryPointRule>

22. Dataflow Entrypoint Rules (Con’t) A rule used to taint parameters going into the method Coverity {  "taint_kind": "servlet",  "tainted_data": {    "all_params_of": {      "and": [        {          "with_annotation": {            "matching": "^org\\.springframework\\.web\\.bind\\.annotation\\.RequestMapping$"          }        },        {          "in_class": {            "with_super": {              "with_annotation": {                "matching": "^org\\.springframework\\.stereotype\\.Controller$"              }            }          }        }      ]    }  }}

23. Dataflow Passthrough Rules A rule used to taint parameters going through 3rd party libraries (no source) Checkmarx Not needed Fortify<DataflowPassthroughRulelanguage="java" formatVersion="3.8"> <RuleID>xxxx</RuleID> <FunctionIdentifier> <NamespaceName> <Pattern>java\.lang</Pattern></NamespaceName> <ClassName><Pattern>String</Pattern></ClassName> <FunctionName><Pattern>format</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> <InArguments>1…</InArguments> <OutArguments>return</OutArguments> </DataflowPassthroughRule>

24. Dataflow Passthrough Rules (Con’t) Coverity public class String {  public static String format(String format, Object args...) {Dataflow.dataflow_return(data);Dataflow.dataflow_return(args);    return unknown();  }}

25. Dataflow Sink Rules A parameter of a function where a vulnerability would occur if untrusted data were passed to it Checkmarx All.FindByMemberAccess(“Statement.execute"); Fortify<DataflowSinkRulelanguage="java" formatVersion="3.8"> <RuleID>xxxx</RuleID><FunctionIdentifier> <Sink><InArguments>0</InArguments></Sink><NamespaceName><Pattern>java\.sql</Pattern></NamespaceName> <ClassName><Pattern>Statement</Pattern></ClassName> <FunctionName><Pattern>execute</Pattern></FunctionName> <ApplyTo implements="true" overrides="true" extends="true"></FunctionIdentifier> </DataflowSinkRule>

26. Dataflow Sink Rules (con’t) A parameter of a function where a vulnerability would occur if untrusted data were passed to it Coverity { "type" : "Coverity analysis configuration", "format_version" : 1, "directives" : [ // Definition of the checker with name, description, remediation { "dataflow_checker_name" : "DF.FAKE_NOSQL_QUERY_INJECTION", "taint_kinds" : [ "network", "servlet", "database", "filesystem", "console", "environment", "system_properties", "rpc" ], "languages": { "Java" : "Webapp-Security-Preview" }, "covlstr_sink_message" : "{CovLStrv2{{t{A tainted value {0} is evaluated by a NoSQLsink.}{\"\"}}}}", "covlstr_remediation_advice" : "{CovLStrv2{{t{Avoid evaluating untrusted data.}}}}" }, // Sinks { "sink_for_checker": "DF.FAKE_NOSQL_QUERY_INJECTION", "sink" : { "param_index": 1, "methods": { "overrides": { "matching" : "com\\.coverity\\.example\\.db\\.FakeDB\\.customSelectorQuery\\(.*" } }}}]}

27. Structural Rules Grep on Steroids. Match based on code structural forms. Checkmarx (finds method declarations of a method named “MyMethod”, only if it is attributed with the “Attr” attribute) All.FindByCustomAttribute("Attr").GetAncOfType(typeof(MethodDecl)).FindByShortName("MyMethod"); Fortify<StructuralRuleformatVersion="3.8" language="java"> <RuleID>yyyy</RuleID> …<Predicate><![CDATA[Variable p: p.enclosingFunction is [Function f: f.parameters contains p] and p.enclosingFunction.annotationscontains [Annotation b: b.type.name matches "org\.restlet\.resource\.(Delete|Put|Post|Get)"] and not p.type.name matches "org\.restlet\.util\.WrapperRequest|org\.restlet\.Request"]]></Predicate></StructuralRule>

28. Hybrid Rules Mix of Structural and Dataflow rules Checkmarx CxList input = All.FindByName("input");CxListdb = All.FindByName("execute");CxListsanitize = All.FindByName("fix"); return db.InfluencedByAndNotSanitized(input, sanitize); Fortify<CharacterizationRuleformatVersion="3.8” language="java"><RuleID>zzzz</RuleID><StructuralMatch><![CDATA[Variable p: p.enclosingFunction is [Function f: f.parameters contains p] and p.enclosingFunction.annotationscontains [Annotation b: b.type.name matches "org\.restlet\.resource\.(Delete|Put|Post|Get)"]]]></StructuralMatch><Definition><![CDATA[ foreachp {TaintEntrypoint(p, {+WEB +XSS}) }]]></Definition></ CharacterizationRule>

29. Coverity Rules Coveritystate machine based checker include matching and events #include “extend-lang.h” enumfun_state_t { UNLOCKED = 0, LOCKED = 1 }; START_EXTEND_CHECKER( block_check, int_store); //name and flow based ANALYZE_TREE() { Fun locking_fun(“lock”); //Define the functions you are looking for Fun unlocking_fun(“unlock”); Fun blocking_fun(“fopen”); if ( MATCH(locking_fun) ) { //When function is called in path trigger event SET_STATE(LOCKED); } else if ( MATCH(unlocking_fun) ) { SET_STATE(UNLOCKED); } else if ( MATCH(blocking_fun) ) { if ( GET_STATE() == LOCKED ) ) { //If calling fopen while locked trigger finding COMMIT_ERROR(“Called fopen within a locking context.”); } } } END_EXTEND_CHECKER(); MAKE_MAIN( block_check)

30. Custom Rules Summary • Why is this important?

31. Custom Rules Sources • Fortify • http://community.hpe.com/hpeb/attachments/hpeb/sws-Fortifyforum/217/1/HP_Fortify_SCA_Custom_Rules_Guide_4.21.pdf • Coverity • https://www.coverity.com/library/pdf/coverity_extend.pdf • Checkmarx • https://checkmarx.atlassian.net/wiki/display/KC/Working+with+Queries • https://checkmarx.atlassian.net/wiki/download/attachments/5406733/CxQL%20API%20Guide%20v8.0.0.pdf?version=1&modificationDate=1458736360763&api=v2

32. You Created the Rules But No Security Findings are Showing Up (What Now?) ???

33. You Created the Rules But No Security Findings are Showing Up (What Now?) Assess results Add custom rules Assess results Scanning & Verification Web Application, Trusted App, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis You have to introduce “Canaries” into your code

34. Canaries • Does anyone know what a “canary” in the code is? • History of Canaries (if needed) • Placement is important • Remember where you placed them as you will need to remove them later • Put them in ever increasing layers of your application • Right near a source of untrusted data • In the first layer of business logic and others at deeper layers of your business logic • Coverage is important • Does anyone know of any good security “canaries” to put into your code?

35. Sample Canaries File path manipulation booleanbusinessLogicMethod(String untrusted_param, … ) { File f = new File(untrusted_param); } SQL Injection booleanbusinessLogicMethod(String untrusted_param, … ) { ...Statement stmt = connection.createStatement();ResultSetrs = stmt.executeQuery(untrusted_param);} Command Injection booleanbusinessLogicMethod(String untrusted_param, … ) {...Runtime.getRuntime().exec(untrusted_param); }

36. If You are Still Not Seeing the Canaries… • Make sure that your rules do not have typos in the method or parameter identifiers • Make sure that you have defined all of the necessary passthrough rules for the 3rd party libraries. • Remember that function calls through packages like java.lang.*, java.util.*, javax.* are third party libraries. • If the Java library function is new it may have been missed by the security researcher

37. Assuming Static Analysis Will Find Everything Assess results Add custom rules Assess results Scanning & Verification Web Application, Trusted App, Mobile Application Add Canaries Assess results Address risk not covered by Static Analysis Manual Code Review Penetration Testing Problems with 3rd Party Libraries

38. Manual Code Review • Authentication • Password Handling (Reset, Storage, and Policies) • Backdoors • Authorization • Input Validation • Session Management • Proper Cryptography • Auditing • Business Logic Vulns • Negative values • Information Leakage • File Upload/Download Logic • Error Handling • Languages not covered well by tool or cross language calls (JNI, Unmanaged Code, system calls, etc.) https://www.owasp.org/index.php/Application_Threat_Modeling#Security_Controls

39. Penetration Testing • Verifies fixes and unresolved issues • Can find things outside the application • Exposed services (S3 Buckets, Github servers, etc.)

40. 3rd Party Libraries (without source) 3rd party library with sinks sends untrusted data to Processing X Sources provided by Application ??? Application Sink X Library Sink • Static Analysis can only find issues in source • Use a tool like OWASP Dependency Check to ensure that you do not have vulnerabilities in your 3rd party libraries • https://www.owasp.org/index.php/OWASP_Dependency_Check • Dataflow analysis may be blocked • Source (or Entrypoint) + ??? (No sinks) = No Vulnerability Found through library

41. 3rd Party Libraries (with source withapplication) 3rd party library with sinks sends untrusted data to Processing Sources provided by Application Application Sink Library Sink Understanding context of libraries Source (or Entrypoint) + Sink = Vulnerability

42. Scanning 3rd Party Libraries (with source withoutapplication) 3rd party library with sinks Processing ??? Library Sink Understanding context of libraries Empty+ Sink = No Vulnerability Found

43. Successfully Scanning 3rd Party Libraries (with source withoutapplication) 3rd party library with sinks Processing Artifically created entrypointson public interfaces of library Library Sink Understanding context of solution Entrypoint rules on public interfaces + Sink = Library Vulnerability

44. Conclusion, Contact Info, and Questions • You should have deeper understanding of static analysis tools and how to get more value out of your tool • Using the techniques in this presentation I have increased result output by over 10X in certain cases for security issues like buffer overflow • Twitter: @KangAbraham • Email: abraham.kang@hotmail.com • Blog: http://abrahamkang.blogspot.com/ • LinkedIn: https://www.linkedin.com/in/abrahamkang • Questions: ?