The challenges of using an intrusion detection system: is it worth the effort?

1. The challenges of using an intrusion detection system: is it worth the effort? Rodrigo Werlinger, Kirstie Hawkey, Kasia Muldner, Pooya Jaferian, Konstantin Beznosov University of British Columbia, Canada Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

2. Werlinger, R., Muldner, K., Jaferian, P., Hawkey, K., Beznosov, K. Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

3. Motivation Literature • “This task was based upon the monitoring and analysis phase of ID, the most time-consuming and cognitively challenging subtask in ID [9, 10, 23]”. • “Command Line or Pretty Lines? Comparing Textual and Visual Interfaces for Intrusion Detection” Thompson et al., CHI 2007 Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

4. Motivation Literature: Our Perception: IDS configuration is *hard* Rodrigo’s current experience deploying an IDS His prior experiences in a telecommunications co. Collective recollections of 1+ interview participants describing IDS configuration as a major hurdle Pre-processing phase of ID is relatively easy “This task was based upon the monitoring and analysis phase of ID, the most time-consuming and cognitively challenging subtask in ID [9, 10, 23]”. Command Line or Pretty Lines? Comparing Textual and Visual Interfaces for Intrusion Detection” Thompson et al., CHI 2007 Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

5. Intrusion Detection Systems (IDSs) • Intrusion detection phases: deployment, monitoring, analysis, response • Still need human intervention • ID requires high level of security expertise, organizational knowledge & collaboration • Most current research focuses on supporting monitoring + analysis phases (e.g., visualization, better detection algs) Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

6. Research questions • What do security practitioners expect from an IDS? • What are the difficulties they face when installing and configuring an IDS? • How can the usability of an IDS be improved? Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

7. Approach • Semi-structured interviews • 9/34 discussed IDS • 6 Academic, 1 Financial Services, 1 Scientific Services, 1 Consultant • 1 Security Manager, 1 IT Manager, 5 security, 2 general IT w/ security duties • Participatory observation • ~15 hours on IDS (~90 total) • Working with 2 senior Academic SPs Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

8. Results from Interviews [an IDS is] “one of the most controversial [tools] – some really love it, but some really hate it” Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

9. IDS Expectations: Advantages • Problem identification • Activities inside/outside firewall • Reduction of uncertainty • Could provide assurance of effectiveness of security measures • Monitoring with privacy • Decreased time pressure for maintenance • If using an Intrusion Prevention system Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

10. IDS Expectations: Disadvantages • Financial expense • Work and time required • Tuning the system • Unreliability • Buggy, dropped packets • Lack of clear utility • Hard to see an improvement, often sit idle Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

11. Results from Participatory Observation • History • IDS installed 2 years prior in one network domain • Crashed, memory space issues • Unclear whether problem was with setup or newly added wireless • No time to confirm exact cause • Decided to re-install from scratch on a different network • This delayed for several months • High workload, competing priorities Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

12. Issues deploying an IDS (1/5) • Improve efficiency of monitoring • But also: • Statistics on network security • Support for increasing security budget • Ultimately, (2) proved too complicated… Deciding on the purpose of the IDS Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

13. Issues deploying an IDS (2/5) • To connect the IDS, 2 ports were needed • Wanted to use port-mirroring feature to select traffic wanted to monitor These requirements could not be realized IDS installed in a less critical network Integrating the IDS in the network Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

14. Issues deploying an IDS (3/5) • Quick tune option • But inadequate for complex task: • Can’t specify hard disk partitions • No support for configuring IDS security settings (server firewall rules) Configuration via IDS GUI Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

15. Issues deploying an IDS (4/5) Distributed Environment • Extra overhead • Involvement of various organizational members without security as a priority • Multiple stakeholders need to configure IDS • But IDS did not support fine-grained access control • Compromise: less critical network, but autonomy Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

16. Issues deploying an IDS (5/5) • Ideally IDS would have been deployed in critical network (utility high, usability low) • Hard to assess IDS utility without full deployment • Unclear if large network domain more demanding • False positives vs. false negatives tradeoff • Can’t tune until running Usability / Utility Tradeoffs Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

17. Challenges throughout IDS deployment Ongoing Use • Collaboration • features • “A bit of smarts” • Reports for different • stakeholders Configuration & Validation Considerations Before Deploying • Show economic • benefit to get buy-in • Minimize overhead • costs (stakeholders) • Broad knowledge of • organization & • systems • Distributed • environment • Initial configuration • hurdle • Determine • appropriate test bed Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

18. Planning • IDSs not yet de facto tools IDS utility must be clear, but until deployed and configured….. • IDS deployment impacts many stakeholders • Formalize via dedicated project • Involve stakeholders Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

19. Configuration and validation • Configuration hurdle (rule customization) • Distributed environment • How to test the IDS (“all or nothing” tool) Quick tuning Flexible reporting Support for finding test-bed Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

20. On-going usage • Detection of trends • “A bit of smarts” • IDS usage via various stakeholders Artificial intelligence Collaboration features Flexible reports Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

21. Summary • Many factors will determine whether deploying an IDS is worth the effort • Challenges are present in all stages and not limited to technology • Tool support needed to help meet the challenges • More study needed to determine generalizability of our participants’ experiences Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

22. Thank youhawkey@ece.ubc.ca Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

23. Challenges and recommendations • Why an IDS? • Perceptions of IDSs • Planning and installation • Broad and deep knowledge • Intensive collaboration • Representative Testebed Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

24. Technical and organizational challenges • Broad and deep knowledge • Intensive collaboration • Representative Testbed • Meaningful reports Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

25. Original slides that came right after the results Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

26. Stages to deploy an IDS Ongoing Use • Collaboration • features • “A bit of smarts” • Reports for different • stakeholders Configuration & Validation Planning • Show economic • benefit • Minimize costs • Detection efficient • Distributed • environment • Initial configuration • hurdle • Determine • appropriate testbed Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

27. Planning IDS not only to detect attacks • Management buy-in • Compare different points in the network Show economic benefit Dedicated project • Involve other stakeholders • Competing priorities • Minimize costs, • Detection efficient Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

28. Configuration and validation Configuration hurdle • Customization of the rules Distributed environment: • How to distribute alarms How to test the IDS • “All or nothing” tool Quick tuning Flexible criteria Find test-bed Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

29. On-going usage Detection of trends • “A bit of smarts” Collaboration features • Incorporate changes in the systems Better reports • Meaningful reports Artificial intelligence Collaboration features Flexible reports Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)

30. Laboratory for Education and Research in Secure Systems Engineering (lersse.ece.ubc.ca)