Loss and Delay Accountability for the Internet

1. Loss and Delay Accountability for the Internet Defense by Jiazhen Chen & Theodore Brockly

2. Motivation • Internet provide no guarantees about when or if packets will be delivered • Enabling flexibility of technologies • Simplify forwarding path • However • To make an informed decision about paths with poor service • Needs to know which domains are currently dropping or delaying its packets • Form of performance feedback would help establish whether providers are performing their duty

3. Probing vs. AudIT • Ping and Traceroute • Reveal how network reacts to probe packets • Fail to capture low-rate or sporadic misbehavior • Hard to be allowed to inspect the internal infrastructure of ISPs • AudIT • Explicit accountability interface • ISPs report their performance through it • ISPs can’t lie regarding their business • Easily done with modification on NetFlow

4. The Goals • Determine which administrative domains are losing or delaying their packets • Provide a traffic source with enough feedback to determine • Guarantee an upper bound on the error that a malicious AD can induce • Tampering should be attributable to a specific link • Do no harm • This idea has contribution to the design of next generation Internet.

5. Threat model • Our threat model does not allow a malicious AD to modify or otherwise tamper with traffic reports from other Ads • They sign legally binding service-level agreements. [Business relationship]

6. Accountability Interface • AudIT definition • Feedback packet • Fields:

7. Accountability Interface • Each AD can choose its own granularity of reporting aggregates • The only restriction is that two aggregates • no packets in common • one is a subset of the other

8. Informal Aggregate and Feedback algebra • Assume two aggregates • If they are combinable: • For a feedback entry – use vector notation

9. Using AudIT • AudIT can be used to provide traffic sources with performance feedback • Honest Reporters • Combine collection of feedback entries on aggregate from multiple Ads to determine AD-level path.

10. Using AudIT with -Honest Reporters

11. Using AudIT with On-Path Lies • On-Path Lies: an AD misrepresents its performance • Detection: • If two ADs’ Feedback entries on the same degree disagree on • How many packets the earlier AD delivered • When it delivered them • Conclude that either one of them is lying, or the inter-link between them is in trouble

12. Using AudIT with On-Path Lies • “Consistent”

13. Using AudIT with On-Path Lies • “Blames” • When feedback entries on the same degree are consistent, an AD can only lie about its performance by implicating one or more of its peers • Y drops the packet but claim it never got them • Y tries to hide delay by claim it received it ms later than it actually did

14. Using AudIT with On-Path Lies • Lemma • If AD X produces correct feedback entries on traffic aggregate , then none of X’s peers can blame any loss or average delay on X with respect to without causing a feedback inconsistency (we omit the straightforward proof for lack of space). • The more information an AD generates about its performance, the more difficult it is for its peers to blame their faults on it.

15. Using AudIT with On-Path Lies • Localization • Exposing lying Ads to the peers they implicated • If an AD responds with a signed entry that differs from the original one, then S concludes that AD is lying • If both Ads insist on their original reports, then S sends both signed entries to both X and Y. • If both tell the truth, they might investigate their inter-AD link • Otherwise, the lying AD is exposed to the peer it implicated

16. Using AudIT with On-Path Lies • Strong Incentive for truth telling • If lying means implicating a peer who will learn that, then lying means entering a dispute with that peer and damaging the corresponding business relationship.

17. Using AudIT with On-Path Lies • Role of Inter-AD links • Feedback inconsistencies between two peering Ads are impossible to properly ascribe when in fact it is the inter-AD link between them that has failed

18. Using AudIT with Off-Path Lies • A malicious A lie about having seen an aggregate when it in fact has not. • When all Ads on an aggregate’s path provide feedback, the AD liar is situated off the actual path of the aggregate

19. Basic Feedback on TCP traffic • AudIT reporting TCP-flow statistics in the number of packets lost and the average delay. • Checkpoints • Inter-AD links / Inside a router / separate box • Places on all links that traffic enters and exits • Roles: Entry Point and Exit Point • Need clocks synchronized • To use NTP or use a GPS receiver with less cost

20. Accountability Center • Part of its network management platform • A Module / Cluster • Depends on the size of AD and the amount of traffic it generates and forwards • Two Interface: • Feedback receiver interface • Follow up interface

21. Packet Classification • A Check point consider packets belong to the same aggregate type if: • All packets have the same Identity Tuple • Any FIN and RST packet is the last one • Flow Records • Maintain short-term state

22. Determine Where to Report • To send feedback to the corresponding source Ads • Feedback-receiver map • First it compiles an IP-prefix-to-origin-AD map using BGP data from border router • Then it looks up the feedback-receiver address for each origin AD and adds that information to the map

23. Determine Where to Report • Tricky scenarios • ASes do not always advertise the address of their router interfaces • Some interfaces are located at exchange points that may be advertised by multiple AS • Non-BGP speaking network has multiple providers • The first two are not relevant to our mechanism • For the third one: • Either do not receive feedback or receive feedback through their BGP speaking providers

24. Long-term StateStatistics Reporting • Each Checkpoints reads its short-term state every Ts seconds • Create feedback entries from its closed records • Packages them per source AD • Copy them to local storage • Send them to the corresponding feedback via UDP • Lost packets can be recovered through the follow-up channel.

25. Follow-up Channel • If receiver misses expected feedback or identifies a inconsistent feedback • Use follow-up interface of the involve ADs to resolve the issue as described previously

26. Limitations • There are two cases, in which providing per-TCP flow statistics is insufficient to characterize AD performance • Split TCP flows • Long, delay-sensitive TCP flows

27. TCP Delay Feedback • A checkpoint considers a sequence of packets to be a single-path TCP flow if: • The packets belong to the same flow • The packets have the same entry and exit points of this AD

28. TCP Delay Feedback • This granularity allows the flow time between ADs, as well as the average packet time per flow to be measured • Need 2 fields to accomplish this: • entryPoint (16)‏ • entryTime (32)‏

29. Statistics Collection • Although conceptually Simple, most ISPs routers aren't equipped to upload time and IP to a capability • Need to find a less intensive solution

30. Statistics Reporting • Workaround Solution: • Each router builds a table of source-destination pairs on a central server, updated periodically • Every Tm seconds, an entry point sends a marker packet to it's destinations • Upon receiving, the end point timestamps and sends the marker to the accountability server

31. Limitations • Multiple entry point flows • Exit points must make an educated guess at the entry point using the lookup table • Cannot compute exact path time for each packet, but can give the general performance of the flow • Reordered packets may cause incorrect time data, may be alleviated by packet annotation at the cost of performance

32. Software Prototype • Emulates Traffic modelling • Netflow-style Statistics • Implements State processing • produces/sends feedback • Two Checkpoints • 3.8 Ghz processors • Emulating 100-checkpoint topology

33. Results • Single Processor handles 10Gbps link • Handles > 250k flows/sec • Assumes 5,000 byte flows • 1 GB memory, 200 GB Storage • Handles 20 million concurrent flows • Long term state for 5h at 1 billion flows/hour • <2% bandwidth overhead • Using 5,000 bytes per flow and 4 ADs

34. Further Discussion • Feedback Tampering • Routers with deceiving timestamps may be a problem in the future • Looking into combating it with Message Authentification Codes, to find corrupt routers • Also looking into the ability to force hop-by-hop feedback. Expensive, but useful for investigation.

35. Further Discussion • Looking into implementing Flow Sampling as a first stage implementation • Reflector Attacks and Spoofing: • Looking into an in depth way to combat Reflector Attacks • Audit can trace the attack to the base AD level, once identified.

36. Conclusion • AudIt: loss/delay accountability • Sources learn per-flow loss/delay within each AD • Accurate in the face of lies • Liars exposed to their peers • Deploying ISPs cannot be falsely accused • Practical to deploy • Reasonable processing/memory req. • <2% bandwidth overhead

37. Thanks~ Any Question?