FEARLESS engineering

1. Reactively Adaptive MalwareWhat is it?How do we detect it?Dr. Bhavani ThuraisinghamCyber Security Research and Education Institutehttps://csi.utdallas.eduThe University of Texas at DallasApril 19, 2013 FEARLESS engineering

2. Outline • Analogies • Malware: What is it? • Our Solutions • Profs. Thuraisingham, Khan, Hamlen, Lin, Makris, Cardenas, Kantarcioglu • Directions • Holistic Interdisciplinary Treatment

3. Analogies: The Human Body • Humans infected with virus and bacteria • Virus replicates itself and spreads throughout the body • Attacks vital organs • Doctor conducts tests and detects the problem • Medicine is given to slow the progression of the disease • Patient’s condition may improve or the patient may die

4. Analogies: An Organization • Bad person joins the organization and pretends to be a good person • He/she monitors what is going on and spies on the organization • Conveys vital information to the adversary – insider threat • Builds a network of bad people • Takes over the organization

5. What is a Malware? • It’s a piece of software that is malicious and carries out bad things • It infects a vulnerable and neglected machine • It attacks the various components of the machine– the operating system (vital organs), applications (limbs) and hardware (bone) • It spreads across a network of machines • It cripples the machines and the network • It conveys vital information to the enemy – the hacker • It takes over the network and carries out its agenda Victim Network

6. What does it look like? Example: Melissa Virus March 26, 1999

7. The Virus-Antivirus Arms Race • Malware (e.g., viruses) • Rogue programs that carry out malicious actions on victim machines • Vandalism (delete files, carry out phishing scams, etc.) • reconnaissance & secret exfiltration (cyber-warfare / hacktivism) • Sabotage (e.g., attacks against power grids) • Randomly mutate themselves automatically as they propagate • Harder to detect since no two samples look identical • Antivirus defenses • Defenders manually reverse-engineer many malware samples • Find mutation patterns • Build defenses to automatically detect & quarantine all mutants FEARLESS engineering

8. Incidents Reported 1990-2001 Everything changed with Code Red attack in 2001

9. Problem is much worse now!

10. Our Malware Team Data Mining Solutions for Malware Professor Latifur Khan Reactively Adaptive Malware and Solutions Professor Kevin Hamlen Android Malware and Solutions Professor Zhiqiang Lin Hardware Malware and Solutions Professor Yiorgos Makris Adversarial Mining Solutions Professor Murat Kantarcioglu Smart Grid Malware and Solutions Professor Alvaro Cardenas

11. Data Mining Knowledge Discovery in Databases Data Pattern Processing Knowledge Extraction The process of discovering meaningful new correlations, patterns, trends and nuggets by sifting through large amounts of attack data, often previously unknown, using pattern recognition technologies and machine learning statistical and mathematical techniques. Data Mining Solutions Thuraisingham, Data Mining: Technologies, Techniques, Tools and Trends, CRC Press 1998 FEARLESS engineering

12. Training and Testing • Extract features • Binary n-gram features • Assembly n-gram features Enhancements to current data mining approaches Hierarchical Clustering (DGSOT) Data Mining Classification Model Training Testing Training Data Good Class Bad Class DGSOT: Dynamically Growing Self-Organizing Tree Our novel solution Testing Data • Supported by US Air Force 2005-2008 • PI: Thuraisingham, Co-PI: Khan FEARLESS engineering

13. Report Results: Example • HFS = Hybrid Feature Set (Binary and Assembly) • BFS = Binary Feature Set • AFS = Assembly Feature Set FEARLESS engineering

14. Reactively Adaptive Malware: What is it? • Next-generation Malware Technology • Malware that mutates NON-randomly • LEARNS and ADAPTS to antivirus defenses fully automatically in the wild • Immune to conventional antivirus defenses • Supported by the U.S. Air Force; 2010-2013 • PI: Hamlen, Co-PI: Khan FEARLESS engineering

15. Data Mining-based Anti-antivirus[Hamlen & Khan] Signature Approximation Model Signature Inference Engine Obfuscation Generation Antivirus Signature Database Signature Query Interface Obfuscated Binary Obfuscation Function Malware Binary Testing propagate

16. “Frankenstein”[Mohan & Hamlen, USENIX WOOT, 2012] • Stitch together code harvested from benign binaries to re-implement malware on each propagation. • Many offensive advantages: • Resulting malware is 100% metamorphic • no common features between mutants • Statistically indistinguishable from benign-ware • everything is plaintext code (no cyphertexts) • Obfuscation is targeted and directed • evolves to match infected system’s notion of “benign” FEARLESS engineering

17. Frankenstein Press Coverage • Presented at USENIX Offensive Technologies (WOOT) mid-August 2012 • Thousands of news stories in August/September • The Economist, New Scientist, NBC News, Wired UK, The Verge, Huffington Post, Live Science, … FEARLESS engineering

18. Solution we are exploring: SNODMAL Stream Based Novel Class Detection D1 D2 D5 D3 D4 C5 C4 C3 C2 C1 Prediction Note: Di may contain data points from different classes D5 D6 D4 Labeled chunk Data chunks Unlabeled chunk Addresses infinite length and concept-drift C5 C4 Classifiers C1 C2 C4 C3 C5 Ensemble FEARLESS engineering • Divide the data stream into equal sized chunks • Train a classifier from each data chunk • Keep the best L such classifier-ensemble

19. Smartphones can also beinfected with malware! FEARLESS engineering

20. Our Solution – Combine Static Analysis with Dynamic Analysis Remote Server • Static Analysis • Data mining solutions • Dynamic Analysis • Platform • Android & I-Phone • Reverse engineering • Level • System call • Operating systems • Network • Supported by US Air Force 2012-2016 • Technical Leads Lin and Khan Network Behavior Mal App App Behavior FEARLESS engineering

21. 3500 counterfeit Cisco networking components recovered The Hunt for the Kill Switch Adee, IEEE Spectrum, 2008 We cannot forget about HardwareDo you Trust Your Chips? Yiorgos Makris(yiorgos.makris@utdallas.edu) Research Supported by: The Hacker in Your Hardware, Villasenor, Scientific American 2010 2012 Phobos-Grunt Mission Fails Due to Counterfeit Non Space-Rated Chips

22. Our Solution to Hardware Trojan FEARLESS engineering

23. That’s not all – Attacks to Critical Infrastructures • Attacks • Maroochy Shire 2000 • Threats Obama administration demonstrates attack to power grid in Feb. 2012 • HVAC 2012 • Stuxnet 2010 • Smart Meters 2012 DHS and INL study impact of cyber-attacks on generator FEARLESS engineering

24. New Attack-Detection Mechanisms by Incorporating “Physical Constraints” of the System • 1st Step: Model the Physical World • 2nd Step: Detect Attacks • Compare received signal from expected signal Physical World Model System of Differential Equations • 3rd Step: Response to Attacks • 4th Step: Security Analysis • Missed Detections • Study stealthy attacks • False Positives • Ensure safety of automated response • [Alvaro Cárdenas, et.al. AsiaCCS, 2011] FEARLESS engineering

25. It never ends!We need to mine the adversary • Adversary changes its behavior to avoid being detected • Data Miner and the Adversary are playing games • Remember, malware detection is a two class problem? • Good class (e.g., benign program) • Bad class (e.g., malware) • Adapt your classifier to changing adversary behavior • Questions? • How to model this game? Does this game ever end? • Is there an equilibrium point in the game? FEARLESS engineering

26. Our Solution: Game Playing • Adversarial Stackelberg Game • Adversary chooses an action • After observing the action, data miner chooses a counteraction • Game ends with payoffs to each player • Adversary may use malware obfuscation • Change has some cost to the adversary • We need data mining techniques to handle the changes by the adversary • Funded by the US Army; 2012-2015 • PI: Kantarcioglu, Co-PI: Thuraisingham FEARLESS engineering

27. Where do we go from here:Holistic Treatment • Three actors interacting with each other: • The Doctor • The Defender/Analyst • The Patient • The User /Soldier • The Virus/Bacteria • The Malware/Attacker An integrated model that represents the behavior of all three actors Together with ECS, SOM, EPPS and BBS, we are proposing an Interdisciplinary approach.