Gödel's Gourd

1. Gödel's Gourd Fuzzing for logic and state issues

2. Introductions • Michael Eddington • CTO and Principal Consultant @ Déjà vu Security • 12+ years in security consulting • Senior developer/architect in prior life • Author of Peach, an open source fuzzer • Device, Kernel, User, Web, Network

3. DARPA Cyber Fast track Thanks Mudge!

4. Defining the Problem • Fuzzers are good at findings implementation issues • …that crash the target • …that are generically detectable (sqli, xss) • Not good at finding design, logic, and state issues • …that do not crash the target • …that are not generically detectable

5. Examples Easy Hard • Buffer Overflows • Memory Corruption • Resource Usage • SQL Injection • Missing authentication • State corruption • Incorrect logic

6. Authentication Examples • Out of 100 admin pages, 5 are missing authentication • Microsoft SSPI skip a step auth bypass • OpenBSD IPSEC incorrect if/then logic

7. Authentication – Detect • Web – Missing Auth • Trigger • Request page w/o logging in • MS SSPI/OBSD IPSEC • Trigger • Skip a step • Status Code (200/403) • What pages require auth • Result (Pass) • Did we complete all steps

8. Logic Example • Windows 95 SMB Flaw • Logic error in password checking code • Length of loop determined by client input • Modified SMB client, ~32 attempsalways wins • We never throw an exception or crash • Typical generic fuzzer will never find this

9. Logic – Win95 SMB boolCheckPw( intuserdata_len, char* userdata, intsysdata_len, char* sysdata ) { for(inti=0; i<userdata_len; i++) if(userdata[i] != sysdata[i]) return false; return true; }

10. Logic – Detect • Win 95 SMB • Trigger • Try all chars • Remove NULL • Result • Does password match

11. State Example • Device (phone/tablet/laptop) with theft system • Agent “heartbeats” to server • Server can trigger “stolen” mode in laptop • Laptop will trigger if unable to “heartbeat” • Timer/counter runs down

12. State – Detect • System Server • Trigger • Cause exception • Flow locked • Unable to heartbeat • Can we perform state flow? • Check result of each step

13. How to detect? • Goal – Modify existing fuzzer to detect these issues • We already produce triggers • How do we add detection?

14. How to detect? • What do we need to detect these issues? • Provide system constraints • If not authenticated result is 402 • If steps 1, 2, and 3 not performed step 4 is error • Result is never 500 • Verify we are still working • Perform state flow w/o mutations

15. Proposed Solution • Gödel's Gourd • Re-use Peach fuzzing engine • Mutation engine • Fault detection/reporting • Constraint language • Control iterations (non mutation iterations) • Mutate state model (skip, order, etc.)

16. Control Iterations • Goal: Verify target is working correctly • No mutations • Constraints pass • State model is followed • Matches recorded control iteration

17. How it works • R – Record iteration • 1 – Fuzzing iteration • C – Control iteration • 2 – Fuzzing Iteration • C – Control iteration • 3 – Fuzzing iteration • … • Remember all states/actions from record iteration • Verify on control iterations • Control iterations every N fuzzing iterations

18. Outcome • If control does not match record – throw fault • Identify conditions that stop normal operation

19. Constraints • Verify logic via simple constraint expressions • Apply constraints to state model • State • Action • Does not modify fuzzer state

20. Language Options • Existing Traditional Languages • JavaScript • Python • Ruby • etc. • Pro • Well known • Available via .NET scripting interface • Cons • Allows modification of fuzzer state.

21. Other Options • Domain Specific Language (DSL) • Use existing • Create our own • Pros • Meet all requirements • Cons • Must implement • Not well known

22. DSL Selection • Object Constraint Language (OCL) • Specification language, no side effects • Developed as part of new UML standards • Familiar syntax • Relatively easy to implement

23. Object Constraint Language (OCL) • Expression types • Invariant (inv) • Always true • Pre (pre) • Evaluated before [ something ] • Post (post) • Evaluated after [ something ] • Can access state from Pre. (@pre)

24. OCL Examples “Car owner must be at least 18 years old” context Car inv: self.owner.age >= 18 “If passwords match result is true” context Login post: result = true implies pass1 = pass2

25. OCL Context • Groups sets of constraints • Constraints for a context are run together • Association based on context

26. Normal Fuzzing Iteration • Enter State Model • State 1 • Action 1.1 • Send Data • Action 1.2 • Receive Data • State N • …

27. Fuzzing Iteration With Constraints • Enter State Model • State 1 • Action 1.1 • Send Data • Action 1.2 • Receive Data • State N • … • Inv(pre) • Pre • EVENT • Inv(post) • Post

28. Applying (Authentication) • Web Authentication # Verify authentication occurred post: (reply = 200 && url.indexOf(‘/admin’) > -1) implies auth.reply = 200

29. Applying (Authentication) • Windows SSPI # Verify all steps completed post: reply = true implies ( auth.step1.reply = true && auth.step2.reply = true && auth.step3.reply = true)

30. Applying (Logic) • Windows 95 Bug post: reply = true implies userpw = ‘password’

31. Applying (State) • Antitheft System • Perform control iteration

32. Implementation

33. Technologies Used • Microsoft .NET Framework – C# • Peach Fuzzer 3 • Cross platform using Mono • OS X • Linux

34. Implementation Diagram

35. OCL Implementation • Irony .NET Language Toolkit • Many differences from traditional • Grammar is code • Easy AST hookups • LINQ Expressions • From IronPythonwork • Last mile is already done

36. LINQ Expressions • Exposes language constructs for use in AST classes. • Does all the heavy lifting. return Expression.Condition( (Expression)ifNode.Evaluate(thread), (Expression)thenNode.Evaluate(thread), (Expression)elseNode.Evaluate(thread));

37. Gödel Usage All the things that do the stuff

38. Peach Pit vs. Gödel Gourd • Data Model • State Model • Agents • Test • Data Model • OCL Definitions • State Model • OCL Associations • Agents • Test

39. Gödel: Define Constraints <Ocl> <![CDATA[ context StatusCodeOk post: context.test.publishers[self.publisher].Result = 'OK' ]]> </Ocl>

40. Gödel: Associate Constraints <Action type="call" method="Logout"> <Ocl context="StatusCodeOk" /> </Action> Constraints will now run with this Action.

41. Gödel: Control Iterations <Test name=“Default” controlIteration=“1”> <Agent … /> <StateModel … /> <Publisher … /> <Logger … /> </Test> Define how often control iterations occur.

42. Usage Feasibility Time and Cost

43. Adding Gödel • Process: • Existing Peach PIT • Add OCL Constraints • Test and Verify Definition • Not recreating full application logic • Just our “view of the world”

44. Time per Protocol • Based on current experience of limited protocol set • Decent in 1 – 2 days • Complete in 1 week or less

45. Performance • What performance impact does Gödel incur? • Constraint evaluation • Control iterations • No performance optimizations…yet

46. Performance of Constraints

47. Performance Control Iterations • Depends on how often, worst case half speed • Never longer than mutation iterations

48. Performance Conclusions • Performance impact dependent on speed of fuzzing • Ability to scale fuzzing lowers impact • For fast fuzzers, acceptable impact • For slower fuzzers, adjust control iterations to occur less often

49. Conclusions • Pentesting/Quick fuzzing • Reasonable for “basics” (verify state’s work, critical logic flows) • General definition building • Reasonable to implement decent coverage • 1-2 days “good enough”

50. Wrapping it up…