CAP6135: Malware and Software Vulnerability Analysis Cliff Zou Spring 2016

1. CAP6135: Malware and Software Vulnerability Analysis Cliff ZouSpring 2016

2. Course Information • Teacher: Cliff Zou • Office: HEC243 407-823-5015 • Email: czou@cs.ucf.edu • Office hour: MoWe 9:45am-11:45am • Course lecture time: MoWe 12:00pm – 1:15pm (Eng1-386A) • Course Main Webpage: • http://www.cs.ucf.edu/~czou/CAP6135-16 • Use the UCF WebCourse for homework submissions, discussion, and grading feedback • Online lecture video stream: • UCF Panopto • Video available in the late afternoon after each lecture • You can access video through the Webcourse “Panopto Videos” tab

3. Prerequisites • C programming language • Software security lecturing will mainly use C code as examples • Programming experience • Any programming language is fine • Knowledge on computer architecture • Know stack, heap, memory • For our buffer overflow programming project • Knowledge on OS, algorithm, networking • Basic usage of Unix machine • We will need to use Unix machine in our department: eustis2.eecs.ucf.edu, for some programming projects

4. Objectives • Learn software vulnerability • Underlying reason for most computer security problems • Buffer overflow: stack, heap, integer • Buffer overflow defense: • stackguard, address randomization … • http://en.wikipedia.org/wiki/Buffer_overflow • How to build secure software • Software assessment, testing • E.g., Fuzz testing

5. Objectives • Learn computer malware: • Malware: malicious software • Viruses, worms, botnets • Email virus/worm, spam, phishing, pharming • Spyware, adware • Trojan, rootkits,…. • A good resource for reading: • http://en.wikipedia.org/wiki/Malware • Learn their characteristics • Learn how to detect, monitoring • Learn how to defend

6. Objective • Learn state-of-art research on malware and software security • Paper reading/presentation for selected milestone papers on related research topics • Face-to-face session students: • Required to do in-class paper presentation • Online students: • Read assigned paper, write paper review

7. Course Materials • No required textbook. Reference books: • 19 Deadly Sins of Software Security (Security One-off) by Michael Howard, David LeBlanc, John Viega • The Basics of Hacking and Penetration Testing (2nd edition) by Patrick Engebretson • Hacker Techniques, Tools, and Incident Handling (2nd edition) by Sean-Philip Oriyano • Online References: • CS161: Computer Security, By Dawn Song from UC, Berkley. • Software Security, by Erik Poll from Radboud University Nijmegen. • Introduction to Software Security, by Vinod Ganapathy from Rutgers • http://www.cis.syr.edu/~wedu/seed/ Hands-on Labs for Security Education, Dr. Wenliang Du, Syracuse University • http://www.hackercurriculum.org/, Guide to ethical hacker publications, • Wikipiedia: Great resource and tutorial for initial learning • Other references as we go on:

8. Grading Guideline Coursework      face-to-face     online streaming • In-class presentation 14% N/A • Paper review reports N/A 14% • Written and lab assignments 20% 20% • Program projects 36% 36% • Term project 30% 30% • We will have two to three programming projects • So you need to have experience in programming!

9. Course Assignment – face-to-face students • Paper presentation • In the later half to 1/3 of the class (when we finish lecturing on knowledge-based content), each class will have two to three face-to-face students present selected milestone papers • Other students are preferred to come to classroom to participate and provide discussion, although it is not mandatory • Occupy about 1/3 to half of the course time • The other time is my lecture time • Only for face-to-face session students

10. Course Assignment – Online students • Write reports on about 10%-15% of presented papers • Provide insight description of the paper’s contribution, explain what YOU think of the paper’s quality, weaknesses, and how to improve the paper’s research

11. Programming projects • Probably will have 3 programming projects • Example: • Basic buffer overflow • Use Unix machine, learn stack, debugger (gdb) • Software fuzz testing • Find bugs in a provided binary program • Network monitoring and analysis • Using Wireshark to analyze captured network traffic

12. Term Project • A research like project • Two students form a group • Or yourself if you cannot find a partner • Will make you do more work • Group format help you to learn how to collaborate • Find topics by yourself • Must related to malware and software security • Provide topic proposal one and half month later • Result: • Submit report at the end of semester • Report will look just like a research paper we read • Face-to-face students: present your project • Online students: submit your presentation slides with speaking notes on every page

13. Questions?