SyFi : A Systematic Approach for Estimating Stateful Firewall Performance

1. SyFi:A Systematic Approach for EstimatingStateful Firewall Performance Yordanos Beyene Michalis Faloutsos Harsha V. Madhyastha Computer Science and Engineering, University of California, Riverside

2. Which firewall will meet my throughput needs? Problem spec: Input: traffic workload Output: select the right firewall Solution requirements: systematic, accurate, cost effective Can't run each workload on each firewall

3. SyFi: Estimating firewall performance without tears Key novelty: - Identify what really affects firewall performance - Develop SyFi predictive model for any workload -- We only need to measure 4 parameters once! SyFi is highly accurate ~94% accuracy - Validated through experiments with real firewalls

4. Motivation: It is the wild west out there.... • Performance is workload specific • There is no systematic methodology • Buyers fliers "lie" by presenting best case numbers Relative order of which is best firewall varies by workload!

5. Roadmap • Previous work • Part I: Measurement: what affects FWall performance • Part II: The SyFi Predictive Model • Validation of Model • Conclusion

6. Part I: What really affects FWall performance? • Experiments conducted on two commercial firewalls • SonicWall E5500, and • Fortinet Fortigate-ONE • We used a third commercial firewall to validate model: • HP TMSzl firewall • Traffic generating tool: BreakingPoint Systems

7. #Concurrent Sessions: no effect • The number of active sessions on firewalls has negligible impact on maximum packet rate of firewalls. • Similar results observed with UDP packets

8. Packet Size: no effect • Packet size does not affect maximum packet rate! • THUS: thruput should be reported in packets/sec • Similar results observed with UDP packets

9. The first packet takes longer:#sessions matters • Packets that create sessions on Firewalls impose significantly higher cost that subsequent packets. • Session rate: • Sessions of 1 packet • Packet ratio: • Session of 10K packets • First packet of a observed with UDP packets

10. TCP is costlier than UDP • TCP packets impose higher cost than UDP packets. • Similar results are observed with UDP versus TCP session packets.

11. SyFi Measurement: ACL Size • Access Control List (ACL) size • no impact on data packets that belong to an existing sessions, • but affects session rate significantly. • Similar results are observed with the other firewall devices.

12. SyFi Measurement: Key Findings • Identified four types of packets that generate different load • TCP SYN packets – trigger session on firewall • TCP data packets • UDP flow first packet - trigger session on firewall • UDP flow subsequent packets • Number of concurrent flows doesn’t matter • Packet size has negligible impact on packet rate • The size of ACL has significant impact on session rate

13. Part II: The SyFi Predictive Model • Measure once, for each firewall, cost Ct: • C1: TCP session start • C2: TCP subsequent packet • C3: UDP flow first packet • C4: UDP flow subsequent packet Cost is expressed relatively to 100% utilization • Transform traffic load to, percentage of packets Pt • #sessions, and length for UDP and TCP • Calculate expected thruput • Note: model does not consider ACL effect

14. Step 1: How to measure packet cost in practice For a given firewall, do Maximum Packet Rate Procedure (MPRP) • Each flow sends 10,000 packets per second • Initialize: start with 1 flow • Repeat: add 1 flow that sends 10,000 packets per second every 60 seconds • Stop: when packets start to drop. • MPRP is used to measure maximum packet rate for both TCP and UDP • The limitation of 10,000 packets /second/flow is test tool limitation

15. Step I: How to measure session setup cost For each firewall, do Maximum Session Rate Procedure (MSRP) • Each flow has ``one” packet • Every second start S new flows • Initialize: start S=5k new flows every second • Repeat: Increase S by 1K flows every 60 seconds • Stop: when packets start to drop For both TCP and UCP sessions • Tip: firewalls configured with low session time-out to avoid session table overflow.

16. Step II: Transform expected workload to numbers • Given that only the four packet types matter • We only need to know • Percentage of packets Pt are in each type!

17. Step III: The Predictive Part Given workload type: • Pt : percentage of packet type t Measured (once per firewall): • Ct : cost of packet type t Two Outcomes: • For given traffic intensity • Predict total cost = system utilization C • Predict max throughput for C = 100% • N: total number of packet/sec of the given workload type • Given the average packet size , throughput can be computed in Bytes per second( bps)

18. Prediction test case Take home message: it is very simple! Sample workload: • 20 % tcp flows, 80% udp flows • tcp flow pkt rate = 10 pkts/second; tcp avg pkt size=512 bytes • udp flow pkt rate = 100 pkts per second; udp avg pkt size=64 bytes; Firewall measured Cost: • C1=1/10000, c2=1/200,000, c3=1/30,000, c4=1/400,000 Model: P1 = 0.2(1/10) , P2 = 0.2(9/10), P3 = 0.8(1/100), P4 = 0.8(99/100) N= 1/(0.2(1/10)*1/10,000 + 0.2(9/10)*1/200,000 + 0.8(1/100)*30,000 + 0.8(99/100)*1/400,000) Throughput in bytes = o.2*N *512 bytes + 0.8*N*64 bytes

19. The model is >94% accurate Workload: • TP1, TP2, TP3, TP4, details in the paper • Compares measured with model results using the third firewall

20. Previous work • Research so far has not focused on this problem, but focuses more on: • Detecting Firewall Rules conflicts • Al-Shaer, E., Hamed, H., Boutaba, R., Hasan, M.: Conflict classification and analysis of distributed firewall policies. In: IEEE JSAC (2005) • Baboescu, F., Varghese, G.: Fast and scalable conflict detection for packet classifiers. In: IEEE ICNP (2002) • Hari, A., Suri, S., Parulkar, G.: Detecting and resolving packet filter conflicts. In: IEEE INFOCOM (2000) • Optimizing firewall rule sets • Acharya, S., Wang, J., Ge, Z., Zane, T.F., Greenberg, A.: Traffic-aware firewall optimization strategies. In: ICC (2006) • Cohen, E., Lund, C.: Packet classification in large ISPs: Design and evaluation of decision tree classifiers. In: ACM SIGMETRICS (2005) • Hamed, H., Al-Shaer, E.: Dynamic rule-ordering optimization for high-speed firewall filtering. In: ASIACCS (2006) • Improving Firewall architecture • Gouda, M.G., Liu, A., Jafry, M.: Verification of distributed firewalls. In: IEEE GLOBECOM (2008) • Gouda, M.G., Liu, A.X.: Structured firewall design. Computer Networks (2007) • Liu, A.X.: Firewall policy verification and troubleshooting. In: ICC (2008)

21. Conclusion Currently assessing firewall performance is “chaotic” Leaves room for manipulation by vendors! Key contribution: - We identify what really affects firewall performance - Develop SyFi predictive model for any workload - We only need to measure 4 parameters once! SyFi is highly accurate ~94% accuracy - Validated through experiments with real firewalls

22. Future Work • Our model was focused on stateful firewalls which inspect packet headers only. • We are working on expanding our model to network security devices that inspect payload. • We expect packet size to have a significant impact on performance when packet payload inspection is involved.

23. Thank you!