1 / 77

Malware Detection based on Application Behavior Modeling

Malware Detection based on Application Behavior Modeling. NWMTD’11 Jun 20–21, 2011. Mrs P.R.Lakshmi Eswari C-DAC, Hyderabad. Evolution of Malware Attacks. Malware Definition (Wikipedia). A software which is designed to infiltrate a computer system without the owner’s informed consent

diem
Download Presentation

Malware Detection based on Application Behavior Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Malware Detection based on Application Behavior Modeling NWMTD’11 Jun 20–21, 2011 Mrs P.R.Lakshmi Eswari C-DAC, Hyderabad

  2. Evolution of Malware Attacks

  3. Malware Definition (Wikipedia) A software which is designed to infiltrate a computer system without the owner’s informed consent Refers to a variety of forms of hostile, intrusive, annoying software code MALicious softWARE

  4. Threat from the Malware • A code • which collects the credit card number or any other personal info • Which makes an application do the buffer overflow and crash • Loosing the private and sensitive information • which shows annoying advertisements without your consent • Which encrypts the data and asks for money to decrypt it

  5. Malware Categories

  6. A Typical Malware Exploit Logic • Motivational Logic • Spam • Data theft • Ransom • Disrupt the routine • Protection Logic • Packing • Anti Debugging • Anti Virtualization Propagation Logic Mails USBs

  7. Attacks - Classified • Untargeted attacks • Attacking websites • Infecting portable storage devices • Attacking social networking websites • Wild malware (worms etc) • Botnets • Targeted Attacks

  8. Targeted Attacks

  9. A Typical Attack Originally a executable Doc file Opens the file, and executes the malware Whenever updates windows, also downloads the malware, sends the data out etc. Malware Changes the windows update program

  10. Botnet IRC Server 4. Attacker will also join this channel (preferably through a program) and issue commands (for e.g. update) 3. Join a channel on IRC Receives the command (update) 1. Exploit / Attack Victim 2. Download malware (bot)

  11. Botnet DDoS (distributed denial of service attacks) Collecting lot of bank related data Spidering attacks (on websites) Spams Using victim for other sensitive attack Shutdown the computer etc

  12. Motivation and Business

  13. Motivation and Business 14

  14. Vulnerability, Exploit and Race

  15. Vulnerability, Exploit and Race

  16. Malware Detection Techniques • Black listing • Anti Virus • Intrusion Detection System • Behavior Based Malware Detection • White listing • Specification Based Detection • Anomaly Detection

  17. Commercial Solutions

  18. End System Security Suites • Centralized configuration on all clients • Centrally controlled • Firewall • Encryption • Device Control • Anti Malware • Security policies

  19. White listing Solutions Core Trace Bouncer Bit9 Parity Robot Genius Microsoft App Locker McAfee Application Control

  20. Don’t want to pay ? ! Free Anti Virus [AVG, AVIRA, AVAST] Free Firewall [Zone Alarm] URL Scanner [AVG, WOT, RG Guard] Trend Micro Web Protection Add on Disable Auto runs Returnil Virtual System / Windows Steady State Wehn-Trust HIPS [MUST for Windows XP – ASLR Tool] Win-pooch HIPS [Windows XP] OSSEC HIDS WinPatrol [BillP Studios]

  21. How anti malware works? Behaviors database Behavior Based Engine (On Process Activities) Basic Activity Scanning * Malware Signature database Anti Virus Scanning (On file content) Known Applications database White listing (On process creation) ( * Process activity, file read or write )

  22. Malware Prevention System (MPS)

  23. MPS - Approach Each application makes sequence of system calls for accessing various OS resources through multiple control paths (normal behaviour) When the application is infected with malware, its behaviour changes User Process n User Process 1 User Process 2 …………….. User Space System Calls Kernel Space Operating System Detects malicious activity before it causes damage to end system i.e. before the system calls are executed by the operating system

  24. MPS - Architecture

  25. Flowchart

  26. Malware Prevention System 1. Application Profiling and Model Generation Process in a Sandbox 4. Client Protection against overall threats - Process Execution Control Model Enforcement Module 2. Server Manages the models and admin can set the policies here Server communication module 3. Based on the policies the model gets pushed to clients 27

  27. Resource - A System calls : {1,2,4} Resource - B System calls: {1,3,4,2} Resource - C System calls: {1,2,4} Model Generation

  28. Operations Hooked in MPS File System Calls Process hooks Network Calls Registry Calls

  29. Deployment Scenario

  30. System Architecture

  31. Database Structure @ Server

  32. Database Structure @ Client

  33. Index File @ Server

  34. Update Request MPS Server MPS Client UPDATE_REQUEST UPDATE_RESPONSE Major No, Minor No, OS type, ModelUpdate, Db Major No, Db Minor No No.of Model Files, Model File names, ModelFile Path

  35. File Transfer Request MPS Server MPS Client TRANSFER_REQUEST TRANSFER_RESPONSE Model File Name with path Contents of the Model File

  36. Log Message Request Application name, OS type, Date, IP, Operation, Path Success or Fail

  37. Client and Server – Technologies used Server on Linux Apache Server 2.2 Virtual Machine Windows XP, Vista and 7 images Linux 2.6.23 kernel image Java runtime environment PHP HTTP message format XML, OpenSSL Windows Client Mini Filter Driver Call out Drivers Win32 programming C, C++ programming PE Executable format Open SSL Linux Client Linux Security Modules C, C++ programming Qt Programming OpenSSL

  38. Server GUI

  39. Client GUI

  40. Malicious Pdf Creation of Axsle.dll Creation of Icucnv34.dll Write file on cvs.exe The malware repeatedly tries to write cvs.exe file and it gets blocked. The document doesn’t open until the write file operation on cvs.exe is completed.

  41. Malicious Pdf

  42. Stuxnet • Behaviors Detected • Hides view of system files • Hidden image file • File has system attribute • Creates logon entry • Unsigned binary • Drops executable • Modifies internet settings • Spawns process

  43. Stuxnet

  44. Stuxnet

  45. ATT27390 doc file • Activities blocked • Dropping of zipfldr.dll in system32 folder • Dropping of wuaueng.dll in system32 folder

  46. Field Testing Report • MPS is compared with similar best commercial tools available in the market like NovaShield, Mamutu, Malware Defender, Sana Security Primary Response, Safe Connect, Threat fire etc.

  47. Field Testing Report • MPS is found sensitive against blended MS office and PDF documents wherein the MPS solution alone identified the malicious activity as the other industry product remain silent • Application has a tendency to raise false alarm against benign documents as it might match the enforcement policies defined • Overall it is felt that the solution is detecting high level targeted malware behaviours, but there is a need to improve the capabilities by suppressing the false alarms.

More Related