Developing a Cyberwar Laboratory and Exercise: Issues and Lessons Learned

1. Developing a Cyberwar Laboratory and Exercise: Issues and Lessons Learned Paul J. Wagner wagnerpj@uwec.edu UW-Stout Information and Cyber Security Workshop 8/24/2006

2. Main Messages • Developing a good cyberwar laboratory and related exercise takes: • Planning • Thought • Resources • Helps to think about goals and structure

3. Main Messages (cont.) • Many issues arise during the development and execution of a cyberwar exercise • Consider and work through as many as possible up front • A few more will arise in spite of your preparation…

4. Laboratory History • Submitted a grant proposal to National Science Foundation (NSF) in late 2002 • Course, Curriculum and Laboratory Improvement (CCLI) program • Adaptation and Implementation (A&I) sub-program • Grant awarded in June 2003 • Three parts • Develop computer security laboratory • Develop two security-related courses • Computer Security • Cryptography and Network Security • Develop course modules for introduction of security issues in other courses

5. Laboratory Goals • Mixed use laboratory • Not enough space to dedicate to security • Need to be able to connect/disconnect from campus network quickly • Support both Windows and Linux • IUP only supported Linux, real-world environment is heterogenous • Be able to emulate a real-world enterprise computing environment

6. Laboratory – Spring 2004

7. One Way to Lower the Cost • Purchase one many-port switch to act as physical switch, all hubs • Can isolate groups of ports • Can bridge groups where needed • Advantages • Significant cost savings • Reduced maintenance need • Disadvantage • Initial setup difficult

8. Spring 2005 (and 2006) Version • Use of Virtual Machines within Physical Machines • Products • Microsoft Virtual PC (used 2005) • Support discontinued for Mac environment in 8/2006 • VMWare (used 2006) • Another possibility: Xen • Operating systems must be modified • Higher performance gained • Layout similar to previous diagram, but only one physical machine needed per station • Bait machines are also virtual

9. Virtual Machines – Pros/Cons • Advantages • Easier to generate a heterogeneous network with a limited amount of hardware • Able to restore virtual machine on any physical machine in lab • Can give student root/administrator privilege on virtual machine • Flexibility in a dual-usage environment • Damage to a virtual machine is a reduced impact • Disadvantages • Size of images (e.g. if saving state across semester) • Time to compress/save • Network bandwidth

10. Ideas for Future • VMWare Player, Server are now freely available • Virtual network as well as virtual machines • Paper on this using UML (another virtualization product) • Storage virtual machines on portable storage (e.g. USB drives, iPods)

11. Laboratory – Physical Issues • Want to provide some sense of physical security for each station • Lab furniture is currently 8 cubicles with high walls • Problem: not good for general usage, students tend to “hide” in lab and take over stations • Future: a more open physical environment?

12. Exercise Overview • Based on exercises attempted and done elsewhere (IUP, US military academies) • Reverse version of “capture the flag” => “plant the flag” • Final exercise in Computer Security course

13. Exercise Overview (2) • Isolated network, consisting of: • Student systems • “Bait” systems (representing businesses) • 8 student teams each given unsecured Windows and Linux systems • 24 hours to secure their systems • 24 hours to locate other systems and plant a flag on as many other systems as possible

14. Student Preparation • Course • Computer Security (CS 370) • Prerequisite – Data Structures (CS 265) • Goals for course • Develop understanding and background in: • Concepts • Tools • Ethics • Issue: ideally would like students to have some networking background • Currently we present this background in course

15. Student Preparation (2) • Approach from perspective of security professional • Learn as defenders of computer systems and networks • Look at what attackers do to understand their mindset and methods • Systems approach in an enterprise environment • Students sign an agreement that stresses ethical issues and behavior, limits their use of tools to scope of course

16. Student Preparation (3) • Weekly Laboratory Exercises • Policies • Ethics, Social Engineering • Information Gathering Tools • Packet Sniffing • Port Scanning • Password Security/Analysis • Vulnerability Assessment • System Hardening • Intrusion Detection

17. The Cyberwar Exercise • Goals • Real-World Project • Team-Based • Focus on Defense in a Realistic Environment • Defense – understand what needs to be done and how to accomplish it • Attack – to experience the mindset and techniques of the attacker

18. Cyberwar Exercise (2) • More Goals • Gain Experience in: • Technological security – with tools used in weekly labs • Physical security • Social security

19. The Cyberwar Exercise (3) • Exercise Structure • Pre-lab • Set up heterogeneous isolated network • Group students into teams • Teams work to prepare, schedule coverage • Teams discover exact environments (shortly before exercise starts)

20. Cyberwar Exercise (4) • Structure (cont.) • Defense Period • Teams secure systems within constraints of exercise • Must keep certain services available; e.g. ssh, mail server • Business is a balance between functionality and security • Students make entries in online log detailing what defensive techniques they’ve used

21. The Cyberwar Exercise (5) • Exercise Structure (cont.) • Attack period • Teams attempt to plant flag on as many systems on network as possible • Defense continues (adjustments, further work) • All activities must be added to online log • Instructor keeps score based on various criteria • Sysadmins attack all student machines at end of period with variety of canned attacks • Discussion • Whole class discussion after exercise completed

22. Scoring Criteria • Positive additions • Number of services up at certain checkpoints • Successful attacks against other machines • Resistance to sysadmin attacks • Quality of log entries • Negative additions • Successful attacks against your machines • Rules violations

23. Laboratory Setup for Exercise • Goals • Heterogeneous and Isolated Network • Same system for each student team • Replicating tool (e.g. Norton Ghost) saves much time • Don’t forget to give each machine its own identity

24. Laboratory Setup (2) • Structure of Isolated Network • One zone (all systems off one hub) • 8 Student Team Systems running older Windows Server, Linux systems • Non-current OSs with known security holes • All tools used in lab exercises • Added several realistic-looking accounts (e.g. backup, logwd, tomcat) with weak passwords

25. Laboratory Setup (3) • Structure of Isolated Network (continued) • Several Non-Student Systems • Other variants of Windows and Linux • 1 Monitoring system • Additional Available Systems • Host systems can be used for internet access

26. Laboratory Setup (4) • Outside software transferred only by “sneaker net” • Reasoning – no automated updates/patches • Students had to understand issues and solutions

27. Major Exercise Issues • Which services to require? • Too few – not realistic • Too many – configuration more complex, difficult to monitor • How much physical access? • Keyboard access allowed? • Problem with student rebooting another system, which hangs waiting for password on BIOS and/or boot loader

28. Exercise Issues (2) • Allow Denial of Service (DoS) attacks? • Realistic, but … • Environment deteriorates • Ethics • Keyboard issue above • Which resources can/should be used?

29. Exercise Experiences • Added accounts were a significant hole • Valid-sounding account names lower the expectation of risk • Non-attended machines were broken into less than the student team machines • Successful teams combined multiple exploits • Combining weak accounts/cracked passwords with buffer overflow exploit

30. Exercise Experience (2) • Social engineering attack showed the power of this method • One student team used spoofed email from instructor to request privileged account on each system with given username/password • Members of half of the teams set this account up • Raised interesting ethical issue re: use of non-class resources

31. Exercise Experience (3) • Must be *very* precise with instructions • Example • Told class could only attack within the laboratory environment • Sysadmin set up log system on regular campus network • Told all teams that log was private, they should report in detail • One team accomplished SQL injection attack on log, gained access to all notes, used this to attack other systems

32. Student Problems / Lessons Learned • Time periods too short for each phase • Suggest extending up to several days for each phase • Exercise too late in semester • Suggested to move it earlier to allow more time on exercise • Students were busy with other final projects, some didn’t participate well

33. Student Problems / Lessons Learned (2) • Not enough student system administration experience • Some had, but others wanted more background on this • Problems with software installation during exercise stemming from lack of knowledge of underlying hardware • Need to document this next time

34. Instructor Problems / Lessons Learned • Not requiring Networking course as a prerequisite meant time spent on networking basics during course, less background to apply to exercise • Tradeoff between wanting to provide an “overview” security course vs. having good background knowledge

35. Instructor Problems / Lessons Learned (2) • Needed to require more available services (e.g. web, db, sftp – now done) • Monitoring exercise is difficult • Continuous physical presence is impossible • Ensuring that student system resources are always available takes forethought • Manual checks, Automated checks • Monitoring all network activity during exercise is difficult • Large quantity of information generated, need to filter

36. Benefits of Exercise • Increased student appreciation of security as a process, not product or state • Issues arise; need to respond • Need to remain continuously vigilant • Increased student appreciation of use of tools • How they can be used by hackers • How they can be used for vulnerability assessment • High level of student enthusiasm!

37. Acknowledgements • Our systems and networking staff, led by Jason Wudi and Tom Paine • It’s difficult to do this well without their support and their help! • Dr. Mary Micco, IUP • Dr. Andrew Phillips • Co-PI on our related NSF CCLI A&I Grant

38. More Information • CLICS – a Computational Laboratory for Information and Computer Security • Development of Physical Lab, Courses, and Modules • More information: http://clics.cs.uwec.edu • Supported by NSF Grant, DUE 0309818 • Paul Wagner, wagnerpj@uwec.edu