網路安全核心技術之現況與發展趨勢

1. 網路安全核心技術之現況與發展趨勢 黃能富教授 高速網路實驗室 清華大學資訊工程學系/通訊工程研究所 E-mail: nfhuang@cs.nthu.edu.tw

2. Agenda • Introduction of Network Security • Content Inspection Technologies • Pattern Matching Algorithms • Flow Classification by Stateful Mechanism • Open Issues

3. -- 駭客無所不在 -- • 2000/3：駭客利用DDos的網路攻擊方式，引起Yahoo、Amazon、CNN、eBay 等知名網站癱瘓 • 2001/7：Amazon.com 旗下的 Bibliofind 遭駭客盜走顧客的信用卡資料 • 2002 中美駭客大戰 • 2003/1 SQL Slammer 攻擊 • 2003/4 大陸「流光」後門程式 • 2003/8 Blaster 疾風病毒攻擊 • 2003/9 SoBig 老大病毒攻擊 • 2003/9 大陸網軍攻擊 • 2004/3 Netsky 天網病毒攻擊 • 2004/4 Sasser 殺手病毒攻擊 • 2005/5 國內大考中心遭駭客竄改資料 • 2005/6 外交部網站遭大陸網軍後門程式竊取外交機密

4. 網路安全的隱憂 • 網路攻擊技術日新月異，攻擊工具易於取得,界面淺顯易懂，不需高深技巧，即可進行攻擊。 • 網路攻擊已不侷限於侵入動作，許多攻擊行為旨在阻斷網站之服務能力。 • 網路通訊設備安全性不足。路由器及交換器僅能檢視封包第三層資訊。 • 防火牆著重在封包第四層資訊檢查。 • 防毒軟體逐漸無法辨識網路攻擊。

5. 網路攻擊工具範例

6. 網路攻擊種類 • Denial of Service (DoS), Distributed Denial of Service (DDoS) • Network Invasion • Network Scanning • Network Sniffing • Torjan Horse and Backdoors • Worm

7. P2P/IM 網安威脅 • P2P (Peer-to-Peer) 分享程式 • IM (Instant Messenger) 即時通 • Spyware 間諜軟體 • Adware 廣告軟體 • Tunneling 私人隧道

8. P2P: A new paradigm • Bottleneck of Server • Powerful PC • Flexible, efficient information sharing • P2P changes the way of Web (Internet)

9. P2P即將破壞現存的資安架構 • P2P 除了檔案分享與即時通訊，也逐漸發展出不同應用，例如 SoftEther 和 Skype。對個人用戶，利多於弊，但對企業，為資訊安全一大隱憂 • P2P 應用潛藏諸多風險，包括 • 洩漏企業內部機密資訊 • 成為病蟲擴散的管道 • 下載非法檔案 • 侵犯著作權 • 佔用大量網路頻寬 • 影響其他系統正常運作 • 造成員工分心，降低生產力

10. Famous P2P Examples • BitTorrent • eZpeer • Kuro • eDonkey • eMule • MLdonkey • Gnutella • Kazaa/Morpheus • Shareaza • Direct-connect • Gnutella • Soulseek • Opennap • Worklink • Opennext • Jelawat • PP點點通 • SoftEther • iMESH • MIB • WinMix • WinMule • Skype

11. Instant Messenger (IM) • MSN • Yahoo Messenger • ICQ • YamQQ • AIM (AOL IM)

12. 網路攻防技術發展趨勢 • IDP/IPS (Layer-7) • Application Firewall (Layer-7) • Network Access Control (NAC) • Defense-in-Depth/Security Switch

13. A Generic Layer-7 Engine • Packet Normalizer • Makes sure the integrity of incoming packets • Eliminates the ambiguity • Decodes URI strings if necessary • Pattern-Matching Engine • Policy Engine • Gather information from pattern-matching engine and issue the verdict to allow/drop the packets

14. Packet Normalizer • Integrity Checking • IP Fragment Reassemble • TCP Segment Reassemble • TCP Segments may come out-of-order • SEQ out of window size • Segment Overlapping • URI Decode • URI hex code obfuscation (‘a’ = %61) • URI unicode/UTF-8 obfuscation • self-referential directories obfuscation (/././././ = /) • directories obfuscation (/abc/a/../a/../a/ = /abc/a)

15. Pattern-Matching Engine • The most computation-intensive task in packet processing. Normally the PM engine needs to process every single byte in packet payload. • In Snort, the PM routine accounts for 31% of the total execution time

16. Pattern Matching is Expensive! • ~50 Instructions/ 1500 Byte packet • ~30 Instructions/ Byte. 45K Instructions/1500 Byte packet Source: Intel Corp.

17. Content Inspection Technologies • Pattern-Matching Algorithms • Software Based • Boyer-Moore • Aho-Corasick (AC) • Wu-Manber • Hardware Based • Bloom-Filter • Reconfigure Hardware (FSM) • TCAM-based

18. Pattern Matching Problem Definition • Given an input text T = t0, t1, …, tn ,and a finite set of strings P = {P1, P2, …, Pr}, the string matching problem involves locating and identifying the substring of T which is identical to Pj = , 1 j r, where ts+i = , 0 i m-1. And this equation can be also denoted as ts…ts+m-1 = Text

19. Aho-Corasick (AC) Algorithm • AC is a classic solution to exact set matching. It works in time O(n + m + z) where z is number of patterns occurrences in T. • AC is based on a refinement of a keyword tree. • AC is a deterministic algorithm. That is, the performance is independent of the number of patterns.

20. An Example of AC Algorithm • Example: P = {ab, ba, babb, bb}

21. An example of AC Algorithm !={h,s} {he} {h} Patterns: hers his she e h r s 0 1 2 8 9 {hers} i {his} s s 6 7 {he, she} h e 3 4 5 {s} {sh} Dashed: fail transitions; those not shown leads to the root

22. e h r s 0 1 2 8 9 i s s 6 7 h e 3 4 5 An example of AC Algorithm i h e s Got a Match! h i s Text: h e i s h i s

23. Reconfigure Hardware (FSM) • Implement the AC FSM in configurable Logic Elements (LEs) of FPGA. • Achieve multiple gigabit performance. (Depends on the FPGA model) • A powerful FPGA is necessary to accommodate thousands of patterns, so that it’s not practical and visible in commercial market.

24. FPGA-based pattern matching • FPGA-based

25. Bloom Filter • Given a string X, the Bloom filter computes k hash functions on it producing k hash values ranging from 1 to m. The same procedure is repeated for all the members of the pattern set. • The input text is verified by generating k hash values in the same way. If at least one of these k bits is found not set then the string is declared to be impossible to match. • Patterns in Length n are grouped into Bn.

26. 1 2 3 4 5 6 7 8 9 … A B C D E F G H I J Payload Stream Bloom Filter (B4) Bloom Filter (B3) B2 B3 B4 Bw …… Bloom Filter (B2) Group signature by length : G4 (X) G3 (X) H1 1 1 1 H2 1 0 1 1 m 0 m m 0 m 0 1 1 H3 1 G2 (X) 1 1 1 Hk Bloom Filter (Cont.) • False positive : • Mim f = (0.5)K, while m = (k x n) / Ln2 • So, totalspace,sum(Bi)= m x (w - 1) • if k = 1,n = 2048, m = 3072 bits k = 1, n= 3072, m = 4608 bits • if k = 4, f = 0.0625 k = 5, f = 0.0313 k = 6, f = 0.0156 K Hash functions H1, H2, …, Hk

27. TCAM fundamental • TCAM stores data with three logic values: ‘0’, ‘1’, ‘X’ (don’t care) • Multiple match modes are needed.

28. Policy Engine • Collect the matching events from Pattern-Matching Engine. • Clarify the relationship between matched patterns: • Ordered: A policy may consists more than one pattern and should be matched in order. • Offset, Depth: The matched position should be within a certain range or location. • Distance, Within: The distance between two matched patterns should be taken into consideration also. • Trace Application States • Some applications are difficult to identify by using only one signature (e.g. P2P). Policy Engine needs to track the connection state like the following diagram: Msg Exchange Data Exchange Request File S1 S0 S2 S3

29. Content Inspection Technologies • Our Pattern Matching Algorithms • Hierarchical Matching Algorithm (HMA) for Intrusion Detection Systems (IEEE Globecom2005) • A Time and Memory Efficient String Matching Algorithm for Intrusion Detection Systems(IEEE Globecom2006) • A Pattern Matching Coprocessor for Deep and Large Signature Set in Network Security System(IEEE Globecom2005) • A Fast Pre-filtering Algorithm for Pattern Matching (IEEE Globecom2006) • Flow Classification by Stateful Methods • IM/P2P Classification

30. Hierarchical Matching Algorithm (HMA) for Intrusion Detection Systems • HMA is a two-tier and cluster-wise matching algorithm • Reduce the amount of external memory access • Reduce the access delay • Reduce the required processing cycle time • Improve the performance of IDS • Low memory requirement • 1.7~63 times better than the state-of-the-art algorithms • Enable an efficient and cost-effective real-time IDS

31. Cluster-wise String Search Narrow Searching Domain Pre-filter Fast Search

32. HMA BM-PH BMH AC-C Memory 326.75 KB 16.013 MB 313.2 KB 439 KB Hierarchical Matching Algorithm (HMA) for Intrusion Detection Systems

33. Pattern Matching Coprocessor for Deep and Large Signature Set in Network Security System System Architecture

34. Pattern Matching Coprocessor for Deep and Large Signature Set in Network Security System Central Control Unit

35. Pattern Matching Coprocessor for Deep and Large Signature Set in Network Security System Simulation Results FPGA Implementation Results

36. Pre-filter: Search Filter Model • All the substrings that filtered by the filter are clear and impossible to contain any of the defined patterns. • And those substrings passed to the pattern matching algorithm may or may not contain pre-defined patterns. • Thus, the search filter may generate false positive but not false negative. • The false positive here refers to the case that a substring without any pre-defined patterns is falsely detected and accepted as with. • An exact string matching mechanism is essential for finding out which patterns are included in the accepted substring.

37. Pre-filter: Search Filter Model

38. Super-Symbol Filter • The basic idea of the proposed Super-Symbol Filter (SSF) algorithm is to treat two bytes data as a super-symbol, and the using of bitmap to indicate the occurrence of each super-symbol in the pre-defined patterns. Match Vector Constructing For example, for the 8-bit ASCII-code, there are 65536 combinations of two bytes data, and a bitmap vector of 65536 entries is used.

39. Filtering phase in SSF-1 Algorithm Input String Text= ABOD CODING IS FOOD Bitmap A B O D C O D F O O D

40. To have better accuracy and less number of false positives, the extended SSF-2 algorithm, two match vectors are employed. The First Match Vector (FMV) is used for the super-symbols being conjugated by the first two symbols in each of the patterns. The Rest Match Vector (RMV) is used for the rest super-symbols in the patterns except those in the FMV. SSF-2 Algorithm

41. The algorithm looks up the FMV and RMV and detects whether the corresponding bit of each super-symbol is 1. Since “AB” and “OD” are not the beginning super-symbol of any patterns (by checking FMV), the filter algorithm only outputs two substrings “COD” and “FOOD”. And only one substring “COD” is false positive in this case. SSF-2 Algorithm

42. Evaluation • To evaluate the scalability and flexibility, the popular Snort IDS signatures are employed. • In case most bits of the bitmap are set as ‘1’, we can expect that the SSF filtering performance will be impacted dramatically as the “hit rate” will be very high. • Fortunately, by tracking the growing paths of Snort rule patterns, the percentage of setting bits for the MV, FMV, and RMV is still very small (less than 5%). Thus, the proposed approaches have a great chance to adopt the fast growth of Snort releases.

43. Performance Parallel Bloom Filter (PBF), Database Processor (IDP) Pentium-4 3.0 GHz personal computer with 1MB level-2 cache, and installed with Intel’s VTune tool

44. The filtering effectiveness of IDP scheme is pretty bad and is not capable to handle Snort’s patterns. This is due to the bitmap used in the IDP scheme has only 256 entries for one byte symbol. And most of the entries of are set as “1” for the Snort’s patterns. Both PBF and SSF schemes are less sensible to the growth of patterns and have a filtering percentage around 80-98%. Filter Percentage & Throughput

45. The PBF is only suitable for hardware-based implementation, the throughput of PBF is less than that of AC. We can see that for the Defcon-1, the system throughput is around double speed-up (270Mbps vs 141Mbps) compared to that of original AC algorithm, and for Defcon-3, the system throughput is even more than three times speed-up (504Mbps vs 153Mbps). The proposed SSF schemes consume far less memory (cache-resident). Filter Percentage & Throughput

46. Flow Classification Using Stateful Method The FA Example : FTP

47. The FAs of BitTorrent protocols.

48. The FAs of Yahoo Messenger protocol.

49. 網路安全研究主題 • DoS/DDoS • Content Inspection Algorithms • Zero-day Attacks • Web Security • Network Access Control (NAC) • Wireless Security

50. Zero-day Attacks MS WMF 0-day exploits 10 Jan, 2006 28 Dec, 2005 29 Dec, 2005 Attack BroadWeb released pattern update MS WMF exploit publicly released Microsoft released patch Microsoft IE creates TextRange() Vulnerability 24 Mar, 2006 26 Mar, 2006 11 April, 2006 Attack Vulnerability was publicly unveiled BroadWeb released pattern update Microsoft released patch