Incident Handling in Academia

1. Incident Handling in Academia What to do when you have been hacked!

2. The Presenters • Scott Fendley • BS Comp Science – U of AR 1999 • MS Comp Science – U of AR 2004 • Security Analyst, Dept of Computing Services • Volunteer Incident Handler, SANS Institute • David Merrifield • Associate Director of Computing Services

3. Session Description • Explores how to handle the attacks on your Internet infrastructure. • Discusses a time-tested 6 step procedure for Incident Handling. • Touches on the legal issues relevant to all Academic Institutions (K12 or Higher Ed) • Dealing with Law Enforcement and handling Evidence • Employee Monitoring vs Student Monitoring

4. Disclaimers, Disclaimers,Disclaimers • I am not a lawyer. Consult your nearest legal counsel if you choose to handle incidents on your campus or have questions. • The majority of this information is the basis of my procedures at the University of Arkansas, but your mileage may vary.

5. Foundation of Incident Handling • An Action Plan for dealing with intrusions, cyber-theft, denial of service and other security-related events • Events can be of a electronic nature or of a physical nature.

6. Definitions • Incident – an adverse event in an information system, and/or network, or the threat of the occurrence of such event. • Ex: unauthorized use of another user’s account • Execution of malicious code • Unauthorized use of system privileges • Event – Any observable occurrence in a system and or/network. • Ex: Packet Traces • System Boot Sequences • Anything that you can record in your IH notebook

7. Incident Handling Metaphor • Incident Handling is like First Aid. • The Handler is under pressure and mistakes can be costly • Practice is a key. Skills degrade without use. • Use pre-designed forms and procedures, and call on others for help.

8. Emergency Action Plan • Remain Calm. • Communicate with your management, and coordinate with your co-workers to keep things focused. • Use formalized language. • EX: Whiskey Five Yankee Mic, We have a bogey on your nine. • Explicit meaning, no room for interpretation is less likely to cause mistakes.

9. Emergency Action Plan • REMAIN CALM (still!) Do not hurry. Mistakes can be costly. • Notes, logs and other evidence are crucial • If the perpetrator is ever found and arraigned, how can you testify if your notes are not organized and detailed? • Failure to take notes is the most common mistake. • Consult your legal counsel for how long you should keep your logs. • Quality not Quantity

10. Emergency Action Plan • Take good notes. • Remember what your English teacher taught you. • The 4 W’s • Who? • What? • When? • Where? • Extra Credit for the 5th W and the H • Why? • How?

11. Emergency Action Plan (1) • Notify your manager of your progress • Do you have easy access to your School’s phone directory? Pager numbers? Home numbers? • If you are over your head, do not hesitate to ask for help • FBI Field Office • handlers@incidents.org • Local Law Enforcement • Trained Computer Forensic Investigators

12. Emergency Action Plan (2) • Enforce a “need to know” policy. • Do not tip your hand to potential insider threats. • Use out of band communications. (Don’t email people about IH discussions.) • Telephones • Faxes • Personal Visits • PGP Keys

13. Emergency Action Plan (3) • Contain the problem. (stop the bleeding) • Pull the network plug? • Pull the power plug? • Forensic Evidence Quandary.

14. Containment Micro Example • Call the user and say “Take your hands off the keyboard and move away from the computer.” • Stand up go to the back of the computer and unplug the network (and/or modem). • Don’t touch anything, we’ll be right there. • Fax instructions/forms for them to fill out.

15. Emergency Action Plan (4) • Make a backup of the affected system(s) as soon as is practical. Use new, unused media. • Make a binary, or bit-by-bit backup. • Failure to make a backup is the second most common error. • Chain of custody of the evidence.

16. Emergency Action Plan (5) • Get rid of the problem. Identify what went wrong if you can. Take steps to correct the deficiencies that allowed the problem to occur. • Nuke the computer or just scrub it? • Get back in business using clean backups and monitor the system to make sure it can resume functioning.

17. Emergency Action Plan (6) • Learn from this experience. • Share your experience with others. • Sys-admin List for K12 • Arktech List for Universities and Colleges • Another useful list is Unisog@sans.org for all Educational entities. • Review the incident from start to completion. • Identify areas of improvement • Engineers versus Mathematicians

18. Seven Deadly Sins of IH • Failure to report or ask for help • Incomplete/non-existent notes • (Accidental) Mishandling/destroying evidence • Failure to create working backups. • Failure to contain or eradicate • Failure to prevent re-infection • Failure to apply lessons learned

19. Emergency Action Plan Summary • Remain calm, don’t hurry. • Notify your oranizations’s management, apply need to know, use out of band communications. • Take good notes (even if you aren’t/can’t prosecute). • Contain the problem • Back up the system(s), collect evidence • Eradicate the problem and get back to business • Lessons Learned

20. Six Steps of Incident Handling • Preparation • Identification • Containment • Eradication • Recovery • Lessons Learned

21. Preparation • Update your organization’s disaster recovery plan to include Incident Handling • Establish visibility and a compensation plan for the team. (Slush fund for food and caffeine for long weekends or evenings of mitigating an emergency.) • Checklists! • Emergency Communications Plan

22. Preparation Key Points • Password Access • Conduct training for incident handlers (War Games) • Establish guidelines for inter-departmental cooperation. • Build relationships with techies and sys admins • Develop interfaces with law enforcement agencies in your area.

23. Small tape recorder Blank Tapes Binary Backup Utils Safe Back Ghost Encase Forensic Software TCT Autopsy Encase Small Hub and cables Laptop (extra batteries) CD’s with clean binaries Sysinternals Foundstone Windows Resource Kit Call List, Phone book Cell Phone (batteries) Fresh Blank Media (CD-Rs Floppys, Zip, etc) Preparation - Jump Bag

24. Policy People Data Software/Hardware Communications Supplies Transportation Space Power and environmental controls Documentation Preparation in a nutshell

25. Identification • Fire Alarm Analogy • Who can pull a fire alarm? • Who authorizes re-entry? • Maintain situation awareness • Provide current “intelligence” • Correlate information (mailing lists are great sources for newest worms/viruses or attacks)

26. Signs of an incident • Intrusion Detection system alarm • Suspicious entries in system or networking accounting • Discrepancies in logs • (Un) successful logon attempts • Unexplained, new user accounts • Unexplained processes or services running • Notification via abuse@ address or phone call • Poor system performance • Unusual time of usage.

27. Identification • Initial Assessment • “Efficient handling of errors is part of the process” • Be careful to maintain a provable chain of custody. • Use the tape record if at all possible to keep notes for you on what commands you run and actions you do. • Make law enforcement sign for any evidence you hand off to them. Assign a value to it.

28. Containment • This is where we cross the threshold in which we begin to actively modify the system. • Keep the system pristine • Pull the system off the network (or perhaps the subnet off the network). • Load your binaries, set the path • Backup the system

29. Containment • Safely store any backup disks/tapes so that they will not be lost and/or stolen. Multiple copies are best with volatile media types. • Keep a low profile. • Analyze a copy of the backup • Report to management on progress • Are you sure you backed up the media in question?

30. Containment • Acquire logs and other sources of information. • Firewalls, IDS Logs • Logs from other systems nearby

31. Containment • Consult with system owners (departmental technical staff) • Change passwords • Determine possible other systems that have potentially had passwords breached. • Packet sniffers are easy to install.

32. Eradication • Is your schools policy to nuke the computer and reinstall with a secured OS, or just clean and secure? • Improve your defenses • Perform vulnerability analysis and system audits. • Locate the most clean backup and carefully install it.

33. Recovery • Restore from backups if required • Be sure you do not restore the malware • Secured system? • Validate the system and create baselines • Test that everything on the system is working as expected with the owner. • Place the final decision on the system owner of when to restore operations. • Monitor the systems

34. Follow-up / Lessons Learned • Develop a follow-up report • Start as soon as possible • Include any forms you used in identification step • Details, details, details! • Lessons Learned Meeting • Executive Summary Report • Recommended Changes to procedures? • Additions to jump kit

35. Legal Issues to Academia • HIPAA • Privacy Rule (2002) • Security Rule (2005) • FERPA (Buckley Amendment) • DMCA • Patriot Act

36. Monitoring • Monitoring employees • Student Privacy • Student-employees?

37. Law Enforcement Contacts • University Police • City Police or County Sheriff • FBI (Field office in LR) • Secret Service • Department of Homeland Security • Infraguard Arkansas

38. More Information • http://www.sans.org/ • http://www.securityfocus.com/ • http://www.foundstone.com/ • http://www.sysinternals.com/ • http://www.incidents.org/ • http://ists.dartmouth.edu/

39. Questions? • Contact me at scottf@uark.edu or call me at 479-575-2022. • Also, talk to those in the state and across the nation for specific questions. • Sys_admin@lists.state.ar.us • Unisog@sans.org