1 / 19

Li-Chiou Chen lchen@pace School of Computer Science and Information Systems Pace University

Identifying Malicious Web Requests through Changes in Locality and Temporal Sequence DIMACS Workshop on Security of Web Services and E-Commerce. Li-Chiou Chen lchen@pace.edu School of Computer Science and Information Systems Pace University May 4 th , 2005.

lorin
Download Presentation

Li-Chiou Chen lchen@pace School of Computer Science and Information Systems Pace University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Identifying Malicious Web Requests through Changes in Locality and Temporal SequenceDIMACS Workshop on Security of Web Services and E-Commerce Li-Chiou Chen lchen@pace.edu School of Computer Science and Information Systems Pace University May 4th, 2005

  2. Needs for anomaly detection in distributed network traces • The fast spreading Internet worms or malicious programs interrupts web services • Early detection and response is a vital approach • These attacks are usually launched from distributed locations • Network traces left at distributed locations are invaluable for searching clues of potential future attacks • E.g. Dshield, the Honeynet Project

  3. Types of IDS • Based on data • Network-based IDS • Monitors and inspects network traffic • Host-based IDS • Runs on a single host • Based on detection techniques • Signature-based IDS • Uses pattern matching to identify known attacks • Anomaly-based IDS • Uses statistical, data mining or other techniques to distinguish normal from abnormal activities

  4. Outline • Toolkits for inferring anomaly patterns from distributed network traces • Previous works • Changes of locality over time • Markov chain analysis • Preliminary results • Summary • Future works

  5. TIAP: Toolkits for inferring anomalous patterns in distributed network traces Network traces (web log, tcpdump, etc) Data conversion Alerts from other IDS or TIAP peers (using IDMEF) Locality pattern analysis Sequence pattern analysis Response module Alerts to other IDS or TIAP peers (using IDMEF) Alerts to administrators

  6. Web level IDS • Anomaly detection • Structure of a HTTP request (Kruegel and Vigna 03) • Normality on streams of data access patterns (Sion et al 03) • Misuse detection • State transition analysis of HTTP requests (Vigna et al 03) • Look for attack signatures (Almgren et al 01)

  7. Changes in locality patterns and temporal sequence patterns • Locality • where the web request is sent, such as the source IP address, • which web server is requested, such as the destination IP address • Temporal sequence • the order of requested objects during a given period of time

  8. Locality pattern analysis in distributed network traces ABAA ABCD KIKL ABPO t1: AB t2: .... t3: …. t4: ….

  9. An example: web traces in common log format from 6 web servers S1 S2 S3 S4 S5 S6 tstamp, ip, server, doc_tpe, user_agent 62978, 38.0.69.1, 1, 2, 3 62979, 38.0.69.1, 1, 2, 3 62979, 38.0.69.1, 2, 2, 3 63001, 38.0.69.1, 1, 2, 3 …….. ……… A session

  10. Data profiles • 6 web servers (2 of them have links to each other, 4 of them are independent) • One day web trace • One session: a distinct IP, 10 minutes interval • 193,070 HTTP requests, • 11,177 sessions • HTTP requests from outside of the organization

  11. Locality pattern analysis 86 sessions by only two web bots

  12. NS N S S O S N S O SN S S N S S………………….. t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t13 t14 t15 t16 ……………. sampling window 1 sampling window 2 N N O S O S Markov chain analysis

  13. Data profiles • 1 web servers • One week web traces • Window size 30 • Reference list 30

  14. Change of distinct IP over time- browsers

  15. Change of distinct IP over time- web bots

  16. Markov chain results 0.43(0.14) Old (O) 0.42(0.21) 0.43(0.17) 0.13 (0.10) 0.13 (0.08) New (N) Same (S) 0.40 (0.22) 0.83 (0.10) 0.06 (0.04) 0.18 (0.16)

  17. Illustration of the state transition probability

  18. Summary • The preliminary locality pattern analysis works well with identifying distinct web bot access patterns • The Markov chain analysis provides a way to infer attacks that utilize random IP addresses • A combination of the two approaches is needed

  19. Ongoing works • Incorporate the analytical results for malware or intrusion detections • A distributed framework of data collection and information sharing for inferring malwares or intrusion attempts across servers/platforms/geographical locations • Collection of attack logs for analytical purpose • Use of the Intrusion Detection Message Exchange Format (IDMEF) for message changes among servers

More Related