1 / 35

Packeteer PacketShaper Update

Packeteer PacketShaper Update. OARNET April 2005. Sean Applegate Mid-Atlantic Territory Engineer (703) 801-0413 Sapplegate@packeteer.com. Agenda. New Features in 7.X DoS Tuning Performance Tuning General Q & A. Several Minor UI Improvements WUI Plugin Architecture Quick Commands

elizabeth-mcguire
Download Presentation

Packeteer PacketShaper Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Packeteer PacketShaper Update OARNET April 2005 Sean ApplegateMid-Atlantic Territory Engineer(703) 801-0413Sapplegate@packeteer.com

  2. Agenda • New Features in 7.X • DoS Tuning • Performance Tuning • General Q & A

  3. Several Minor UI Improvements WUI Plugin Architecture Quick Commands IPFilters Flow Detail Records (FDR) Adaptive Response Technology (ART) What’s new in 7.X

  4. User Interface Improvements

  5. Quick Commands – Making Many Changes Quickly 2 3 4 1 5

  6. IPFilter Command PacketShaper# ipfilter show Show IP info clear Clear IP filter[s] onlyaccept Only accept specified packets passthrough Passthrough specified packets discard Discard specified packets iponly Relay only IP trafic on|off. Is applied in the NIC so it is more efficient than the Ignore/Discard policies.

  7. IPFilter Command Usage: ipfilter passthrough <device> src|dst <ipaddr> [<mask>] PacketShaper# ip passthrough main outside src 192.168.32.1 PacketShaper# ip passthrough inside dest 192.168.32.1

  8. IPFilter Command PacketShaper# ip show Relay all traffic. Exclude Filters: total 2 [19] hits 0 Inside dst 192.168.32.1 (ffffffff) --> passthru [20] hits 3 Outside src 192.168.32.1 (ffffffff) --> passthru Include Filters: total 0 PacketShaper# ip clear clearing all filters...

  9. Off-box per-flow records Architecture is Similar to NetFlow but w/more application & performance data points Typically increases CPU usage by 5-10% Sends UDP Packet with up to 32 Flow Records to Flow Collector Collectors ReportCenter Linux OpenSource (Perl/PostgreSQL) Any NetFlow-5 Collector Formats NetFlow-5 Packeteer-1 Packeteer-2 Flow Detail Records (FDR) Go to Documentation

  10. FDR Architecture Third_Party Billing Software Cisco Collector ReportCenter NetFlow v5 NetFlow v5 Packeteer-1/2

  11. Packeteer-1

  12. Packeteer-2

  13. Packeteer-2 cont.

  14. Implementing FDR PacketShaper# setup flowrecords id 1 packeteer-2 192.168.32.25 9292 on < Linux PacketShaper# setup flowrecords id 2 packeteer-2 192.168.32.50 9800 on < RC3 PacketShaper# set flowrecords show ID RecordType CollectorIP Port Enabled 1 packeteer-2 192.168.32.25 9292 on 2 packeteer-2 192.168.32.50 9800 on 3 4 PacketShaper# setup flowrecords id 2 off < Turn off entry 2 PacketShaper# setup flowrecords id 2 none < Remove entry 2 Go to Documentation

  15. AR Overview - What is Adaptive Response? • Adaptive Response is Packeteer technology that leverages its network visibility to inform, alert, or implement practical controls in response to a change in network conditions • AR uses information available in the Measurement Engine, Hostdb and specific commands to trigger on a selected condition. • AR has several components: • Templates • Agents • Action Files • Incident Reports • Goal: Enable the design of a Smarter Network that can respond automatically to common issues. Go to Documentation

  16. AR Overview – Templates • Template – building blocks for AR • Each template identifies the metrics that can be monitored for a specific resource. • Options: • Resource that is monitored. • Metric(s) that are available to measure thresholds against. • Variables available to reference in action files. • General • Template Specific

  17. AR Agents • Agent – The Glue • Multiple agents can be based on a single template • Specifies the exact metric to monitor • Specifies the resource to monitor (link, class, host, etc) • Specifies Green, Yellow and Red Thresholds • References optional Action files which implement alerts or actions. • Creates Incident Report when triggered Go to Documentation

  18. AR Overview – Action Files & Drilldowns • Action files (also called command or .cmd files) • Provides the commands to run when an agent triggers • Example - Send an email when inbound/default becomes greater than 15% of the partition bandwidth. The action file may look something like this for the Inbound Default Traffic agent: partition apply $ClassName 15% fixed send email dblanco@packeteer.com ‘$agentname agent’ ‘The $ClassName class has exceeded 15% of the link for more than 60 minutes. It has been limited to 15% of the link. Go to the Info tab to view the Incident Report for $agentname agent to see more details.’ • Drilldown (also called Incident Report) • Produces detailed information on the events that caused an agent to trigger • On the WUI, holding the mouse over the AR category produces a window with a list of links to drilldown files or graphs – clicking on the link shows the drilldown in a separate window • Example - High Bandwidth Host drilldown looks like this: "time","agent3","avg-bps","host-ip","direction" "06-Jan-2005 14:10:00","High Bandwidth Host",13816,10.10.253.99,outbound "06-Jan-2005 14:10:00","High Bandwidth Host",7816,10.1.1.27,inbound Go to Documentation

  19. Default ART Templates • Unit Health • Unit Limits • System Load • Memory Allocation • Network Health • Traffic Performance • Partition Utilization • App Health • Default Traffic • New Application • High Bandwidth New App • Host • High Bandwidth Host • Host Info Variables • Failed Flow Ratio • New Flows Per Minute (NFPM) Failed Flow • New Flows Per Minute (NFPM) Side Unknown • General • Class ME Variables • Partition Variables • Link ME Variables

  20. AR Dashboard

  21. AR Page

  22. Editing an Agent

  23. 1 2 3 Adding an Action File 5 4 Go to Documentation

  24. DoS Tuning • What to Look For? • Load Shedding • Policy Flow Limit • AR Agents • NFPM Failed w/discard policy (~500 failed flows/min) • Class ME Variables • TCP Inits (/inbound & /outbound) • TCP Server Ignores (link inbound & outbound) • TCP Server Ignores by class (wild card /inbound/* & /outbound/*) • Actions – Enable FDR, Block Hosts, Enable PacketCapture, Run Forensic Commands • Forensic Analysis • Hostdb info –sp –n 25 • Hostd info –sp <network> <mask> • Hostdb info –sf –n 25 • Hostdb info –sr –n 25 • Tr h f <host> or Tr h r <class path> • Flow Detail Records

  25. What to Look For During a DoS Attack • Host flow failures • Hostdb info –sp –n 10 • Tcp-inits • Tcp-server-ignores • Tcp-server-ignores% • Client-floodblocks • Server-floodblocks • Tcpallocfailures • Hostdballocfailures • ReportPortal has quick access to all these metrics • Analysis Matrix > Failed Flows, Traffic History • Report Portal > Advanced > I/O Troubleshooting Reports • Configure Events or AR Agents to monitor above normal hits.

  26. Load Shedding • Drop new and/or existing connections on a per host basis that exceed a define threshold • White List for Exception Hosts Go to Documentation

  27. Implementing Load Shedding PacketShaper# setup loadshedding exception add list:exceptionhosts PacketShaper# setup loadshedding clientFPM 2000 PacketShaper# setup loadshedding serverFPM 1000000 PacketShaper# setup loadshedding FailedFPM 1000 PacketShaper# setup loadshedding enable PacketShaper# setup loadshedding show Load Shedding: Enabled Shed New Flows Only: Enabled ========================================================================= Load Shedding Parameters Current Default Min Max ========================================================================= Client FPM 5000 1000000 10 1000000 Server FPM 1000000 1000000 10 1000000 Failed FPM 1000 1000000 10 1000000 TCBConn 100 100 5 1000000 UCBConn 100 100 5 1000000 Load Shedding Host Exception List: exceptionhosts

  28. Policy Flowlimit • Similar to Load Shedding but done in PacketWise Software and takes into account all flows and can only effect new flows. • The limits are set to default values of 10,000 flows per minute on client hosts and 100,000 flows per minute on servers. • Flows limits include flows of any type, not types of just that traffic type. • policy flowlimit <tclass> none|<client-fpm> <server-fpm> Go to Documentation

  29. Hostdb Info PacketShaper# hostdb info -sp -n 5 IP Address Conn RTT Cur 1 Min Peak --- New Flows Per Minute --- to PS rate avg rate Client Server Failed -------------------------------------------------------------------------------- 137.45.62.218 I 1054 1ms 10k 13k 1.3M 248 6 174 137.45.86.84 I 104 1ms 4248 4552 1.2M 165 7 140 137.45.68.31 I 145 1ms 4209 7057 1.4M 166 6 128 219.133.4.180 0 --- 0 20k 1.6M 113 0 105 137.45.68.40 I 25 1ms 3672 2928 1.8M 106 68 101 Go to Documentation

  30. Tr h r <host> / Tr h f <class> PacketShaper# tr h r -----( /Inbound/Denied/Soulseek )----- 12-Apr-2001 22:12:40 Soulseek 137.45.62.218 4979 68.36.231.123 1235 pcp09971203pcs.narlington.nj.comcast.net Go to Documentation

  31. Performance Tuning • The Basics • Hard Code Interfaces • Prune Your Traffic Tree – smaller trees are faster • Increase discovery thresholds (Setup Tab > System Variables) • Turn Discovery on Sparingly (schedule it) • Classify/Shape Resnet, Admin, Library, Labs Separately • Use Partitions and Policies Together • Use Rate Policies on Medium to Long Lived Flows • Don’t Use Rate Policy on /__bound/Default classes • Protect and Limit Traffic – all too often higher ed just limits traffic • Use Scheduling as Appropriate > Backup Configs, ToD Shaping, etc. • Don’t run your device at ‘Red Line’ thresholds • Advanced • Configure AR System Health Action Files and ME Failure Metrics • Create a Customer Portal with Key Performance Reports • System Health • System Limits • Sys kmem <pkt | com> types

  32. System Health Current % Idle = 100 Average % Idle = 99 (last 25 secs) Minimum % Idle = 0 Vector 1 ( 25958, 6126, 77697, 102998) Vector 2 ( 548994, 243126, 184608) Vector 3 ( 1037898, 830514, 987846) Load % = 0 (1000 ticks) Random Ppt = 0 0 Hard Ppt = 0 0 Shedding Ppt = 0 0 No Buf Ppt = 0 0 TxDone Queue = 0 0 TxPend Queue = 0 0 Rx Queue = 0 0 TxDone Bkup = 0 0 ticks Rx Queue Dly = 0 0 ticks Overhead = 0 0 ticks

  33. System Limits Statically allocated objects Current Remaining Total ------------------------------------------------------------------- Traffic classes 201 1847 2048 Partitions 24 1000 1024 Dynamic Partitions 13504 6496 20000 Policies 71 1977 2048 Matching rules 509 4613 5122 Classes with worst clients/servers 0 16 16 Classes with top talkers/listeners 11 1 12 TCP flows 34845 365155 400000 Other IP flows 14923 185077 200000 Legacy flows 8 19992 20000 Concurrent Hosts 200000 0 200000 Represents entries in hostdb, not active hosts. We over-write old hosts so once it is has seen the max hosts it will stay there until reset.

  34. Sys kmem commands – packet engine / common processes Idx Type Blocks Allocs Fails Bytes HiWater Limit Pcnt 34 magic 8959 729016725 0 322524 1433160 12582912 3.88 35 appness 5510 443470843 0 286520 307944 12582912 3.45 47 discovery 0 0 0 0 0 6291456 0.00 50 congestion 74897 -1260946167 641326241 6291348 6291516 6291456 75.70 52 appness-tracking 0 0 0 0 0 6291456 0.00 53 classThingy 0 0 0 0 0 6291456 0.00 55 dshash 0 0 0 0 0 6291456 0.00 56 crumbs 1790 -1934452076 0 161176 796176 6291456 1.94 59 dynamic-ptn 0 0 0 0 0 6291456 0.00 61 logger 0 0 0 0 0 6291456 0.00 80 appness hash 49 280865 0 20436 68568 6291456 0.25 81 appness peeker 5510 443305107 0 198360 213192 12582912 2.39 82 appness data 2051 153351844 0 302188 409728 12582912 3.64 83 appness criteria 5510 1299790183 0 693888 906096 18874368 8.35 84 appness utils 149 11687607 0 6548 162836 6291456 0.08 85 appness misc 366 4026244 0 26984 244636 6291456 0.32 86 flow records 0 0 0 0 0 6291456 0.00 ==================================================================================== 0 104795 267180072 641326241 8310976 10850116 0 100.00

  35. Performance Sizing Feature Requests Support / Resources Training Futures Etc General Q & A

More Related