Architectural Support for Security in the Many-core Age: Threats and Opportunities

1. Architectural Support for Security in the Many-core Age: Threats and Opportunities Dr. Nael Abu-Ghazaleh and Dr. Dmitry Ponomarev Department of Computer Science SUNY-Binghamton {nael,dima}@cs.binghamton.edu

2. Multi-cores-->Many-cores • Moore's law coming to an end • Power wall; ILP wall; memory wall • “End of lazy-boy programming era” • Multi-cores offer a way out • New Moore's law: 2x number of cores every 1.5 years • The many-core era is about to get started • Will have more cores than can power -> likely to have a lot of accelerators, including the ones for security • How to best support trusted computing? • Critical to anticipate and diffuse security threats

3. Security Challenges for Many-cores • Diverse applications, both parallel and sequential • New vulnerabilities due to resource sharing • Side-Channel and Denial-of-Service Attacks • Performance impact is a critical consideration • Can use spare cores/thread contexts to accelerate security mechanisms • Can use speculative checks to lower latency

4. Defending against attacks on shared resources

5. Attacks on Shared Resources • Resource sharing (specifically the sharing of the cache hierarchy) opens the door for two types of attacks • Side-Channel Attacks • Denial-of-Service Attacks • Our first target: software cache-based side channel attacks. • First, some cache background...

6. Background: Set-Associative Caches

7. L1 Cache Sharing in SMT Processor Instruction Cache Issue Queue PC Register File PC PC Execution Units Data Cache Fetch Unit PC PC PC PC Load/Store Queues PC LDST Units Decode RegisterRename Re-order Buffers ArchState Private Resources Shared Resources

8. Last-level Cache Sharing on Multicores (Intel Xeon) 2 × quad-coreIntel Xeon e5345(Clovertown)

9. Advanced Encryption Standard (AES) • One of the most popular algorithms in symmetric key cryptography • 16-byte input (plaintext) • 16-byte output (ciphertext) • 16-byte secret key (for standard 128-bit encryption) • several rounds of 16 XOR operations and 16 table lookups secret key byte Lookup Table index Input byte

10. Cache-Based Side Channel Attacks • An attacker and a victim process (e.g. AES) run together using a shared cache • Access-Driven Attack: • Attacker occupies the cache, evicting victim’s data • When victim accesses cache, attacker’s data is evicted • By timing its accesses, attacker can detect intervening accesses by the victim • Time-Driven Attack • Attacker fills the cache • Times victim’s execution for various inputs • Performs correlation analysis

11. Attack Example Cache a b c d Main Memory AES data Attacker’s data b>(a≈c≈d) Can exploit knowledge of the cache replacement policy to optimize attack

12. Simple Attack Code Example #define ASSOC 8 #define NSETS 128 #define LINESIZE 32 #define ARRAYSIZE (ASSOC*NSETS*LINESIZE/sizeof(int)) static int the_array[ARRAYSIZE] int fine_grain_timer(); //implemented as inline assembler void time_cache() { register int i, time, x; for(i = 0; i < ARRAYSIZE; i++) { time = fine_grain_timer(); x = the_array[i]; time = fine_grain_timer() - time; the_array[i] = time; } }

13. Existing Solutions • Avoid using pre-computed tables – too slow • Lock critical data in the cache (Lee, ISCA 07) • Impacts performance • Requires OS/ISA support for identifying critical data • Randomize the victim selection (Lee, ISCA 07) • Significant cache re-engineering -> impractical • High complexity • Requires OS/ISA support to limit the extent to critical data only

14. Desired Features and Our Proposal • Desired solution: • Hardware-only (no OS, ISA or language support) • Low performance impact • Low complexity • Strong security guarantee • Ability to simultaneously protect against denial-of-service (a by-product of access-driven attack) • Our solution: Non-Monopolizable (NoMo) Caches

15. NoMo Caches • Key idea: An application sharing cache cannot use all lines in a set • NoMo invariant: For an N-way cache, a thread can use at most N – Y lines • Y – NoMo degree • Essentially, we reserve Y cache ways for each co-executing application and dynamically share the rest • If Y=N/2, we have static non-overlapping cache partitioning • Implementation is very simple – just need to check the reservation bits at the time of replacement

16. NoMo Replacement Logic

17. NoMo example for an 8-way cache • Showing 4 lines of an 8-way cache with NoMo-2 • X:N means data X from thread N

18. Why Does NoMo Work? • Victim’s accesses become visible to attacker only if the victim has accesses outside of its allocated partition between two cache fills by the attacker. • In this example: NoMo-1

19. Evaluation Methodology • We used M-Sim-3.0 cycle accurate simulator (multithreaded and Multicores derivative of Simplescalar) developed at SUNY Binghamton • http://www.cs.binghamton.edu/~msim • Evaluated security for AES and Blowfish encryption/decryption • Ran security benchmarks for 3M blocks of randomly generated input • Implemented the attacker as a separate thread and ran it alongside the crypto processes • Assumed that the attacker is able to synchronize at the block encryption boundaries (i.e. It fills the cache after each block encryption and checks the cache after the encryption) • Evaluated performance on a set of SPEC 2006 Benchmarks. Used Pin-based trace-driven simulator with Pintools.

20. Aggregate Exposure of Critical Data

21. Sets with Critical Exposure

22. Impact on IPC Throughput (105 2-threaded SPEC 2006 workloads simulated)

23. Impact on Fair Throughput (105 2-threaded SPEC 2006 workloads simulated)

24. NoMo Design Summary • Practical and low-overhead hardware-only design for defeating access-driven cache-based side channel attacks • Can easily adjust security-performance trade-offs by manipulating degree of NoMo • Can support unrestricted cache usage in single-threaded mode • Performance impact is very low in all cases • No OS or ISA support required

25. NoMo Results Summary (for an 8-way L1 cache) • NoMo-4 (static partitioning): complete application isolation with 1.2% average (5% max) performance and fairness impact on SPEC 2006 benchmarks • NoMo-3: No side channel for AES, and 0.07% critical leakage for Blowfish. 0.8% average(4% max) performance impact on SPEC 2006 benchmarks • NoMo-2: Leaks 0.6% of critical accesses for AES and 1.6% for Blowfish. 0.5% average (3% max) performance impact on SPEC 2006 benchmarks • NoMo-1: Leaks 15% of critical accesses for AES and 18% for Blowfish. 0.3% average (2% max) performance impact on SPEC 2006 benchmarks

26. Extending NoMo to Last-level Caches • Side-channel attack is possible at the L2/L3 level, especially with cache hierarchy that explicitly guarantee inclusion • Attacker can invalidate victim’s lines in L2/L3, thus forcing their evictions from private L1s. • Effect of partitioning is much more profound at that level. • Have to address the possibility of a multithreaded attack. • Examine other designs for protecting L2/L3 caches. Latency is less critical there. • Investigations in progress...

27. Using extra Cores/threads for security

28. Using Extra Cores/Threads for Security • Main opportunity: using extra cores and core extensions to support security: • Improve performance by offloading security-related computations • Reduce design complexity • Applications that we consider: • Dynamic Information Flow Tracking (DIFT) • Dynamic Bounds Checking (not covered in this talk)

29. Dynamic Information Flow Tracking • Basic Idea: • Attacks come from outside of processor • Mark data coming from the outside as tainted • Propagate taint inside processor during program execution • Flag the use of tainted data in unsafe ways

30. Security Checking Policies • Memory address AND data tainted • Load address is tainted • Store address is tainted • Jump destination is tainted* • Branch condition is tainted* • System call arguments are tainted* • Return address register is tainted • Stack pointer is tainted • Memory address OR data tainted*

31. Existing DIFT Schemes and Limitations • Hardware solutions: • Taint propagation with extra busses • Additional checking units Limitations • intrusive changes to datapath design • Software solutions: • More instructions to propagate and check taint Limitations • High performance cost • Source code recompilation

32. Hardware –based DIFT add r3,r1,r4 WriteBack t 0 0 + data 1 t r1+r4 data 1 r1 InstructionDecode data 1 r0 0 add r3 r1 r4 data 1 r4 r1 r1 1 1 0 ALU r2 1 0 ExceptionCheckingLogic MEM r3 r1+r4 1 0 r4 r4 0 0 r5 0 IFQ RF 1 Taint Computation Logic Existing DIFT

33. Software –based DIFT add r3 r1 r4 add r3 r1 r4 compiler WriteBack add r3 r1 r4 shr r2 r5 2 shl r0 r5 1 data and r2 r2 16 data and r0 r0 16 InstructionDecode r0 data or r0 r2 r0 data r1 or r5 r5 r0 ALU r2 00111100 r3 MEM r4 01100000 A small region of memory is used to store taint information of memory IFQ RF r5 is used for storing taint information of remaining register file Existing DIFT

34. Our Proposal: SIFT (SMT-based DIFT) • Execute two threads on SMT processor • Primary thread executes real program • Security thread executes taint tracking instructions • Committed instructions from main thread generate taint checking instruction(s) for security thread • Instruction generation is done in hardware • Taint tracking instructions are stored into a buffer from where they are fed to the second thread context • Threads are synchronized at system calls

35. Instruction Flow in SIFT

36. SMT Datapath with SIFT Logic

37. SIFT Example add r3 r1 r4 IFQ 1 add r3r1 r4 WriteBack 0 0 1 data 1 r1+r4 1 data + or 1 data add r3r1 r4 SIFT Generator r1 r1 1 data InstructionDecode r0 0 0 ALU 0 r1 1 1 r1 r4 r4 MEM r2 1 r3 r1+r4 0 1 or r3 r1r4 r4 0 0 r4 r5 0 RF 1 RF 2 or r3 r1 r4 IFQ 2 Shared Resources Context-1 Context-2 DIFT

38. SIFT Instruction Generation Logic 2. Secutiry Instruction Opcodes are read from COT 1. Taint Code Generation 3. Rest of the instructions are taken from Register Organizer and stored Instruction Buffer 4. Load and Store Instruction’s memory addresses are stored in Address Buffer

39. Die Floorplan with SIFT Logic • SUN T1 Open Source Core • IGL synthesized using Synopsys Design Compiler using a TSMC 90nm standard cell library • COT, IB and AB implemented using Cadence Virtuoso • The integrated processor netlist placed and routed using Cadence SoC Encounter • Cost 4.5% of whole processor area

40. Benefits of Taint Checking with SMT • Software is not involved, transparent to user and applications (although the checking code can also be generated in software) • Hardware instruction generation is faster than software generation • Additional hardware is at the back end of the pipeline, it is not on the critical path • No inter-core communication

41. Number of Security Instructions per committed Primary Thread Instruction

42. SIFT Performance Overhead

43. SIFT Performance Optimizations • Reduce the number of checking instructions by eliminating the ones that never change the taint state. • Reduce data dependencies in the checker by preloading taint values into its cache once the main program encounters the corresponding address • Reduce the number of security instructions depending on taint state of registers and TLB

44. Eliminating Checking Instructions Primary Thread SIFT Security Thread SIFT – F Security Thread lda r0,24176(r0) xor r9,3,r2 addq r0,r9,r0 ldah r16,8191(r29) ldq_u r1,0(r0) lda r16,-26816(r16) lda r0,1(r0) lda r18,8(r16) extqh r1,r0,r1 sra r1,56,r10 bne r2,0x14 and r9,255,r9 stl r3,32(r30) ldl r2,64(r2) lda r16,48(r30) bic r2,255,r2 bis r3,r2,r2 bis r0,r0,r0 bis r9,r9,r2 bis r0,r9,r0 bis r29,r29,r16 ldq_u r1,0(r0) bis r1,r0,r1 bne r1,0xfffffffffffff080 bis r16,r16,r16 bis r0,r0,r0 bis r16,r16,r18 bis r1,r0,r1 bis r1,r1,r10 bne r2,0xfffffffffffff080 bis r9,r9,r9 bis r3,r30,r3 bne r3,0xfffffffffffff080 stl r3,32(r30) ldl r2,64(r2) bis r2,r2,r2 bne r2,0xfffffffffffff080 bis r30,r30,r16 bis r2,r2,r2 bis r3,r2,r2 bis r9,r9,r2 bis r0,r9,r0 bis r29,r29,r16 ldq_u r1,0(r0) bis r1,r0,r1 bne r1,0xfffffffffffff080 bis r16,r16,r18 bis r1,r0,r1 bis r1,r1,r10 bne r2,0xfffffffffffff080 bis r3,r30,r3 bne r3,0xfffffffffffff080 stl r3,32(r30) ldl r2,64(r30) bne r2,0xfffffffffffff080 bis r30,r30,r16 bis r3,r2,r2

45. SIFT Logic with Instruction Elimination

46. Performance Impact of Eliminating Security Instructions Performance Loss of SIFT and SIFT-F compared to Baseline Single Thread execution Percentage of Filtered Instructions

47. Performance Impact of Cache Prefetching

48. SIFT Datapath with TLB Based Optimization

49. SIFT Logic with TLB Based Optimization

50. Details of TLB Based Optimization • For Stores • tainted register -> clean page – generate instructions • tainted register -> tainted page – generate instructions • clean register -> clean page – don't generate instructions • clean register -> tainted page – generate instructions • For Loads • tainted page -> tainted register – generate instructions • tainted page -> clean register – generate instructions • clean page -> tainted register – generate instructions • clean page -> clean register – don't generate instructions