Ranger Update

1. Ranger Update Chuck Boeheim SLAC Ranger – Chuck Boeheim

2. History • Nicemon • Patrol 2.0 • Ranger 3.0 – Never released • Ranger 4.0 Ranger – Chuck Boeheim

3. New Approach • Previous versions interpreted a rules file • This added complexity and errors • New version is a Perl toolkit • No scanner, parser, or interpreter needed • Rich library of tests and actions provided • Nearly anything else can be expressed in Perl Ranger – Chuck Boeheim

4. Observations • An observation is an object with the values describing one entity • E.g. one process, one filesystem, one file • Observations may be saved between runs • Deltas calculated from previous runs • History may be accumulated • Observations may be made about aggregate entities, too • E.g. cumulative processes for one user, numbers of daemons running. Ranger – Chuck Boeheim

5. Collectors • Collectors return observations to the calling script • Also responsible for saving them between calls • New collectors easily created by overriding the base Collector class Ranger – Chuck Boeheim

6. Built-in Collectors • PSCollector – processes • DFCollector – file systems • FileCollector – file sizes, md5 signatures • NSCollector – listening sockets • SystemCollector – load averages, uptime, number of users, processes • Watch – tail a file or process (in development) Ranger – Chuck Boeheim

7. Includes code for collector Basic Structure Generates one observation per call use PSCollector; apply pscollector => ruleset { rule { cpu > 50 and time > 60*MIN } action { log; mail LIMIT, user; kill }; rule { cpu > 50 and time > 20*MIN } action { log; mail HOG, user; nice }; }; Calls collector repeatedly, feeds observation to ruleset Brackets a set of rules Rules return true or false Actions are blocks done when true The first rule that succeeds ends the ruleset Ranger – Chuck Boeheim

8. Rules • A rule statement is followed by a block that must return true or false rule { (pct > 50 and delta > 10) or (pct > 75 and delta > 5) } sub chk { pct > $_[0] and delta > $_[1] } rule { chk(50,10) or chk(75,5) } • Each rule has an associated action Ranger – Chuck Boeheim

9. Actions • An action statement is followed by a block. • Any perl statements allowed within the block. rule { pct > 90 } action { system 'cleanup' }; Ranger – Chuck Boeheim

10. Built-in Actions • Built-in actions operate on the current observation: • nice – kill – restart • mail – page – log • mcons • action { kill; mail RUNAWAY, user; page BADPROC, 'admin', log }; Ranger – Chuck Boeheim

11. Messages • The Message statement stores text in a library and tags it: Message HOG => <<'EOF'; You have been using $pct% of one CPU on $host. EOF • Actions like mail, page, and log retrieve message by their tag: mail HOG, user; • Messages are evaluated each time they are retrieved. Ranger – Chuck Boeheim

12. Observation Values • Functions provide values from current observation • name – size – user • pct – time – delta • loadavg – numcpu – numusers • Numprocs • Can be used in either rules or actions. • Prototyped unary functions need no args Ranger – Chuck Boeheim

13. Scheduling Tasks every 10*MIN => background { apply pscollector => ruleset { … }; }; Ranger – Chuck Boeheim

14. Units • Constants are provided to provide natural expression of units: • Time: SEC, MIN, HOUR, DAY • Size: KB, MB, GB • Examples cpu > 2*HOUR size > 1*MB and delta > 100*KB Ranger – Chuck Boeheim

15. Repetition Suppression • Many actions take a parameter to suppress multiple triggers for the same observation mail HOG, user, 6*HR; Ranger – Chuck Boeheim

16. MD5 Watching • The file collector can observe MD5 checksums of files • Either supply a list of known good checksums: rule { name eq '/lib/security/pam_afs.so' and md5change('bf1501489fc0bb9bf0052f624558aed4', '13f50f924bbe64758c6400c4fd412ae7') } • Or let it record the first one as the base: rule { name eq '/usr/lib/libc.a' and md5change } Ranger – Chuck Boeheim

17. Filtering a File • Intended to replace swatch for many purposes apply watch('/var/log/messages') => ruleset { rule { /error/ } action { mcons; log }; rule { /ecache parity error/ } action { page …}; }; Ranger – Chuck Boeheim

18. Example: Watch for Bad Guys apply pscollector => ruleset { # Processes that we want to know about. rule { name =~ /crack|irc|eggd|satan/ } action { mail PROC_REPORT,'boeheim',1*DAY; mcons ALERT; log 1*DAY }; } Message PROC_REPORT => '$name is running on $host'; Ranger – Chuck Boeheim

19. Example: Watch Daemons apply pscollector([sshd ools]) ruleset { # Only execute this ruleset for summary records by name. return unless summary eq 'BYNAME'; rule { number == 0 } action { my $mailto = admin; perform ruleset { rule { name eq "sshd" } action { restart '/usr/etc/sshd' }; rule { name eq 'ools' } action { $mailto = 'oomonitor' }; }; mail NO_DAEMON, $mailto; mcons ALERT; log 6*HR; } } Ranger – Chuck Boeheim

20. Example: Watch Filesystems apply dfcollector() => ruleset { rule { name =~ m(^/(var|tmp)) } action { rule { size 99 } action { mail FSFULL, admin, 6*HR; log 2*HR }; rule { size 95, [90,1], [80,5], [50+50] } action { mcons 2*HR; log 2*HR }; }; }; Ranger – Chuck Boeheim