CAP6135: Malware and Software Vulnerability Analysis Cliff Zou Spring 2013

1. CAP6135: Malware and Software Vulnerability Analysis Cliff ZouSpring 2013

2. Course Information • Teacher: Cliff Zou • Office: HEC243 407-823-5015 • Email: czou@cs.ucf.edu • Office hour: MoWe 12:00pm-2:00pm • Course lecture time: MoWe 10:30am – 11:45am (BA 110) • Course Main Webpage: • http://www.cs.ucf.edu/~czou/CAP6135/index.html • Use the new UCF Canvas for homework submissions, discussion, and grading feedback • Very similar to previous webCourse. • Login at: https://webcourses2c.instructure.com/ • Online lecture video stream: • UCF Tegrity • http://tegrity.ucf.edu/ • Recorded via my own Tablet PC in face-to-face sessions • Video available in the early evening after each lecture

3. Prerequisites • C programming language • For our software security programming projects • Knowledge on computer architecture • Know stack, heap, memory • For our buffer overflow programming project • Knowledge on OS, algorithm, networking • Basic usage of Unix machine • We will need to use Unix machine in our department: eustis.eecs.ucf.edu, for programming projects

4. Objectives • Learn software vulnerability • Underlying reason for most computer security problems • Buffer overflow: stack, heap, integer • Buffer overflow defense: • stackguard, address randomization … • http://en.wikipedia.org/wiki/Buffer_overflow • How to build secure software • Software assessment, testing • E.g., Fuzz testing

5. Objectives • Learn computer malware: • Malware: malicious software • Viruses, worms, botnets • Email virus/worm, spam, phishing, pharming • Spyware, adware • Trojan, rootkits,…. • A good resource for reading: • http://en.wikipedia.org/wiki/Malware • Learn their characteristics • Learn how to detect, monitoring • Learn how to defend

6. Objective • Learn state-of-art research on malware and software security • Paper reading/presentation for selected milestone papers on related research topics • Face-to-face session students: • Required to participate in presentation of assigned papers, in-class discussion • Online students: • Read assigned paper, write review • Comment on in-class student’s presentation • Your evaluation will feedback to presenter!

7. Course Materials • No required textbook. Reference books: • Building Secure Software: How to Avoid Security Problems the Right Way  by John Viega, Gary McGraw • Software Security: Building Security In (Addison-Wesley Software Security Series) (Paperback) Gary McGraw • 19 Deadly Sins of Software Security (Security One-off)  by Michael Howard, David LeBlanc, John Viega • Hacking: The Art of Exploitation, 2nd Edition by Jon Erickson • Reference courses: • CS161: Computer Security, By Dawn Song from UC, Berkley. • Software Security, by Erik Poll from Radboud University Nijmegen. • Introduction to Software Security, by Vinod Ganapathy from Rutgers • Wikipiedia: Great resource and tutorial for initial learning • Other references as we go on:

8. Grading Guideline • Coursework      face-to-face     online streaming • In-class presentation      20%                     N/A • In-class participation     10%                     N/A • Paper review reports      N/A                       30% • Homework                    10%                      10% • Program projects            30%                      30% • Final term project            30%                     30% • Right now we have two programming projects ready. If we add the third programming project, the their weight will probably be higher.

9. Course Assignment – face-to-face students • Paper presentation • Each class will have two students present two selected milestone papers • Students are required to participate and provide discussion • Discussion will count in your grade! • Occupy about 1/3 of the course time • The other 2/3 time is my lecture time • Only for face-to-face session students

10. Course Assignment – Online students • Write reports on about 30% of presented papers • Provide comments on student presentation in your reports • Enforce online students to watch video • Collected/Anonymized comment feedback be accessible to everyone • A great help to improve student presentation • Even if you are not the presenter

11. Programming projects • Probably will have 3 programming projects • Example: • Basic buffer overflow • Use Unix machine, learn stack, debugger (gdb) • Software fuzz testing • Find bugs in a provided binary program • Internet worm propagation simulation • Or network intrusion detection experiment

12. Term Project • A research like project • Two students as a group • Or yourself if you cannot find a partner • Will make you do more work • Group format help you to learn how to collaborate • Find topics by yourself • Must related to malware and software security • Provide topic proposal one and half month later • Result: • Submit report before semester ends (late April) • Report will look just like a research paper we read • Face-to-face students: present your project • Online students: submit your presentation slides with speaking notes on every page

13. Questions?