DERBI : D iagnosis, E xplanation and R ecovery from B reak- I ns

1. DERBI: Diagnosis, Explanation and Recovery from Break-Ins Mabry Tyson Pauline Berry Nate Williams Doug Moran David Blei Artificial Intelligence Center SRI International 333 Ravenswood Avenue Menlo Park CA 94025 http://www.ai.sri.com/~derbi Tyson@AI.SRI.COM 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

2. DERBI Objective • Assist SysAdmin after an attack • No special security expertise required • Detailed system analysis as though by a OS/security expert • For sites that didn’t think they needed a real-time ID system • Require nothing beyond off-the-shelf OS • No special logging or monitoring • Provide guidance on what happened and how to recover • How much info can be detected after-the-fact? 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

3. System Description • Rules specify bits of evidence and associated exploit • Rule Graph embodies relationships of evidence and attack goals • Beliefs of evidence combined to generate overall belief of attack • Anthropomorphic characterization of system • Head - High level control • Body - Passes messages between Head and Feet • Feet - Runs around and does the work 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

4. Head • Uses PRS (Procedural Reasoning System) • Operates on rule graph • Goal is to determine whether attack happened • Goal is achieved by acquiring evidence • Handles user interaction • User can add evidence • Rules can query user • Results presented to user • User can drill down 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

5. Body • Allows Head to deal with abstract queries • Allows Feet to deal with O/S specific queries • Deals with multiple hosts • Network communications • Time differences • File system differences 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

6. Feet • O/S specific • Knows how to traverse file system • Careful to collect file info before altering it • Understands special file locations • Parses log files • ID Evaluation primarily exercises the Feet • Solaris & Linux • Only Solaris used in ID Evaluation 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

7. Example Evidence Rule:EJECT buffer overflow EVIDENCE-TYPE (exploit (setuid root) buffer-overflow) UNIQUE-NAME eject-1 EVALUATION-NAME eject PATHS (follow-links '("/usr/bin/eject")) EVIDENCE ( ((not (and (command-version-vulnerable-p DIR FILE) ;; not vulnerable command or (window-of-opportunity (TimeAccessed PATH)))) ;; not used in interval of interest 0 0) ;;; assign 0% probability to command being used and 0% believe that it was ((greater-than (TimeAccessed PATH) ;;; use is later than (max (TimeModified "/cdrom") (TimeModified "/floppy"))) ;;; expected effects 40 100)) ;;; 40% probability of exploit, no change in believe about whether it was exploited POSIT ((posit ((TIME (TimeAccessed PATH))) (compromised-shell "root" TIME *unknown-time*))) EXPLANATION (next slide) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

8. Evidence Rule:EJECT buffer overflow (cont) UNIQUE-NAME eject-1 PATHS (follow-links '("/usr/bin/eject")) EXPLANATION (explain-evidence ( PATH;;; variable declarations (TIME (print-unix-time (TimeAccessed PATH))) (TIME2 (print-unix-time (TimeModified "/cdrom"))) (TIME3 (print-unix-time (TimeModified "/floppy"))) ) (TimeAccessed PATH) ;;; “as-of” time "The command ~S is version vulnerable to a buffer overflow attack and appears to have been used at time ~A which is more recent than two associated files: /cdrom (~A) and /floppy (~A)." PATHTIMETIME2TIME3) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

9. Time: 08-Apr-1999 13:11:57 EDT Exploit: Suspicious-login (Suspicious-login) Login was found for user "doireano" from host 194.27.251.21. This user not seen before. ------------------------------------------------------------ +00:12:05 later Time: 08-Apr-1999 13:24:02 EDT Exploit: FORMAT (FORMAT-1) The command "/usr/bin/fdformat" is a version vulnerable to a buffer overflow attack and appears to have been used at time 08-Apr-1999 13:24:02 EDT which is more recent than the associated device: "/devices/sbus@1f,0/SUNW,fdtwo@f,1400000:c,raw" (04-Mar-1999 11:52:23 EST). +00:02:17 later Time: 08-Apr-1999 13:26:19 EDT Exploit: Unauthorized/nonstandard file activity (FILEACT) 1 files were created with no obvious legitimate user having access. Root users currently are *None*. Normal users are (erink doireano ulandusm grzegors). Groups with a member logged in are *None*. Ignored logins are *None*. Groups with an ignored login are *None*. Files' owner: root Files's group: staff Protection: -rw------- /.sh_history Example Output for an Attack 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

10. Checking a Suspect System DERBI DERBI DERBI DERBI 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

11. Data Sources for ID Evaluation • File system is only source of information • System files • Log files • File system • DERBI has capability to query operator • For example, compare file to backup version • Allow operator to indicate remote login normal or suspicious 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

12. Target System Configuration Files • Passwd • Notes crackable passwords • Hosts.equiv, .rhosts • Notes capability for passwordless logins • Notes world-writable system directories • Crontab files • Notes programs run from crontab 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

13. Log Files • utmpx, wtmpx, utmp, wtmp, lastlog • All compared for inconsistencies • Note logins without logouts • Note inconsistencies in tty usage • Note currently unknown users • Note remote logins from a new host for that user • Note failed logins 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

14. utmp wtmp syslog utmpx wtmpx messages authlog cronlog crontabs Shell Init Files Filesystem Log File Information Relationships • Partial redundancy of info • Redundancy a common result of the evolution & growth of systems • Use to check for tampering • Also exposes changes to system clock lastlog sulog 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

15. Log Files (2) • Syslog, messages, authlog • sendmail messages (mailbomb, locally sent mail) • su times • sshd messages (failures, successful logins/logouts) • ntp anomalies • Verify time of log messages monotonic 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

16. File System Info • Executables • Access times usually means execution • Comparison of suid execute-time vs data file access time • Checksums checked for vulnerable or replaced versions • Normal files • File access/creation, owner and protection recorded for every file • Files that indicate login/logout are specially noted (dot files, pty and window system files) • Special files • Known cracker file names (included deleted files) • Rarely used files that crackers may use 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

17. Evidence Correlated by Time • File access/creation and log information sorted by time • Unauthorized access detected when no authorized user known to be logged in at time files accessed or created • Complications: • Background processes, servers and scheduled jobs • Suid executables • Attacks usually evident by clustering of evidence • Often see evidence of an exploit • Followed by evidence of unauthorized access to files • However, attack can be inferred from a single anomaly 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

18. Detection of New Attacks • “New attack” means new exploit • DERBI spots the intentional and secondary effects of the cracker on the system, after the (new) exploit • Crackers often leave a large trail of evidence • Exploit files touched • Camouflage attempts often leave footprints • Data collectors & back doors often detectable • However, ID Evaluation attacks often are hit-and-run 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

19. Detectable Attacks • Detects R2L, U2R, Data attacks on Solaris (and Linux) • Can detect some DoS attacks when logged (mailbomb, ssh, or telnet attempts) • Generally can only detect latest use of executables (i.e., only the last eject attack could be detected) • Cracker or normal activity can destroy evidence of attack • Can’t detect network traffic but not blinded by encryption 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

20. ID Evaluation Results • Test procedure artifacts complicated evaluation • Evaluation team affected file system (apparently including running attacks) outside of simulation runs but with clock set to times within simulation periods • Dot files accessed and files written in a user’s directory but simulation contained no login • Executables such as eject accessed without device accessed as though an attack was done, but no attack at that time during simulation • Also overwrote access times of all files on some days • Simulated “attacks” were often just exercise exploit and leave • DERBI picks up evidence of usage of privileges 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

21. ID Evaluation Results • 25 attacks in detectable classes • 17 attacks detected • score of 16.98 (68%) • 47 false alarms • score of 25 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

22. ID Evaluation Results - Misses • 8 misses • 1 attack missed due to test procedure overwriting access times • ffbconfig • 5 attacks left no evidence • guessftp, xsnoop, xlock,httptunnelusage (x2) • 2 attacks indistinguishable from normal activity • httptunnel setup - no recognizable suspicious indications • ps -telnet from a new host, but otherwise nothing suspicious 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

23. ID Evaluation Results - False Alarms • 47 total false alarms (total score of 25) • 29 probably due to test procedure (total score 15.2) • 18 definite test procedure artifacts (score 4.55) • 11 probable test procedure artifacts (score 10.65) • 18 other false alarms (total score 9.8) • 7 pseudo-tty errors (looked like log file truncation) (score 5.1) • 5 login/logout record problems (score 3.6) • 3 dot files accessed when user not logged in (score 0.03) • 2 root accessed secret files in a sweep of file system (score 1) • 1 secret access while logged in locally and remotely (score 0.05) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

24. ROC - Overall Total Attacks: 25 Hits: 17 (16.98) Total FAs: 47 (25) Hits: 18 (17.98) Total FAs: 18 (9.8) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

25. ROC - Old vs Overall Total Attacks: 23 Hits: 15 (15) Total FAs: 47 (25) Hits: 16 (16) Total FAs: 18 (9.8) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

26. ROC - R2L Total Attacks: 12 Hits: 6 (6) Total FAs: 2 (1.7) Hits: 6 (6) Total FAs: 1 (0.7) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

27. ROC - U2R Total Attacks: 11 Hits: 9 (9) Total FAs: 21(18.45) Hits: 10 (10) Total FAs: 10 (7.5) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

28. ROC - Data Total Attacks: 3 Hits: 3 (2.98) Total FAs: 26 (6.53) Hits: 3 (2.98) Total FAs: 8 (2.28) 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

29. DERBI Project Ends • DERBI has come to its end -- for now • Experience at analyzing intrusions as a sysadmin led to the idea a system could be built to do this and to make it easier for less experienced sysadmins 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

30. DERBI is a Success • Successful at detecting intrusions on a stock system • Original idea of a post-mortem analysis has been proven • Designed for real intrusions, it performs better the more the cracker does • Difficult to imagine how to further improve detection without modifying O/S 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

31. DERBI is Different • The DERBI concept is orthogonal to most other ID systems • This diversity could be useful as the systems have different strengths and weaknesses • Didn’t fit too well with the design of the ID evaluation • Not a substitute for intrusion monitoring systems, but can aid those sites that don’t want the overhead of such systems 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting

32. Parting Thoughts • The problem of intrusions has a variety of responses for a variety of consumers • Read-only systems or network computers • Brick-up-the-door approach • “We can’t let it happen” approach (most IDS) • “It happens” approach (DERBI) • ID shouldn’t be an after-market add-on to an OS • Watch for incoming and outgoing attacks 1999 Intrusion Detection Evaluation Joint DARPA ID/SIA PI Meeting