1 / 36

TRODS Transparent Recovery for Object Delivery Services

Presented at DSN-DCCS 2011 in Hong Kong on 6/28/11. TRODS Transparent Recovery for Object Delivery Services. Wyatt Lloyd , Michael J. Freedman Princeton University. Service. Server. Client. Server. Connection Recovered!. Server. Server. Object Delivery Services. Read-Only

harva
Download Presentation

TRODS Transparent Recovery for Object Delivery Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Presented at DSN-DCCS 2011 in Hong Kong on 6/28/11 TRODSTransparent Recovery forObject Delivery Services Wyatt Lloyd, Michael J. Freedman Princeton University

  2. Service Server Client Server Connection Recovered! Server Server

  3. Object Delivery Services • Read-Only • Static Content • Webpages, Images, Videos

  4. Work Now • Can’t Modify Clients

  5. Key Idea • Coerce client to help • To identify connections that need recovery • To reliably store information • Yet client is unmodified and unaware • Exploit TCP spec to control client’s stack

  6. Object Delivery Cluster Service Liveness Monitor Server Server Load Balancer Server Server

  7. Failure Service Liveness Monitor Server Server Load Balancer Server Server

  8. TRODS Service Liveness Monitor ? Server Load Balancer Client Server Server

  9. TRODS Service Liveness Monitor ? Server Load Balancer Client Server Store Server

  10. Road to Recovery Step Technique Redirect to live server ………………. Liveness monitor updates load balancer Induce client to send packet……… Coerce client’s TCP stack Continue Connection Determine Phase………………… Use packet + stored info Identify Object……………………. Stored Info Find Offset ………………………….. Use packet + stored info New New

  11. Coercing Clients • Always Leave A Packet Unacknowledged Retransmit Queue Retransmit Queue Exploit TCP Spec for Recovery Initiation! Client Server ACK SYN ACK FIN/ACK Request ACK ACK Response3 SYN/ACK FIN Response1 Response2 ACK Request FIN/ACK SYN Response1 Response2 Response3 FIN SYN/ACK Always Something Here

  12. Continuing the Connection • Determine Phase: • TCP Setup • HTTP Setup • HTTP Download • TCP Teardown TRODS Saves Info

  13. Continuing the Download • HTTP ObjectID • Offset = TCP Ack – HTTP ObjectISN HTTP ObjectISN TCP Ack SYN HTTP Resp Header HTTP Object TCP ISN

  14. Continuing the Download • HTTP ObjectID • Offset = TCP Ack– HTTPObjectISN HTTPObjectISN TCP Ack SYN HTTP Resp Header HTTP Object TCP ISN

  15. Persistent Store • Key-Value Store + Corner Cases Handled + Unlimited Objects • Still Efficient (1 save only) • TCP Timestamp + Very Efficient (1 machine only) • 1 Million Object Limit • Corner Cases KV IP Payload TCP TS Exploit TCP Spec for Persistence! 18

  16. Recover the Connection • Initiate New Connection • GET ObjectID … • Range: bytes=Offset- • Splice Connections Together • Works with Unmodified Servers!

  17. TRODS IP IP TCP TCP’ … … • Packet Manipulation Server TCP TRODS IP

  18. TRODS • Packet Manipulation • Protocol Inspection Server Response1 TCP TRODS IP Request ObjISN ObjID Request

  19. TRODS • Packet Manipulation • Protocol Inspection • Blocks Connection Server TCP TRODS IP ObjID ObjISN Response1

  20. TRODS IP IP TCP TCP … … • Packet Manipulation • Protocol Inspection • Blocks Connection • State Injection Server TCP TRODS IP TS

  21. TRODS • Packet Manipulation • Protocol Inspection • Blocks Connection • State Injection • Recovery Initiation Server TCP TRODS IP ? Ack

  22. Failure Walkthrough Service TCP TCP TCP Liveness Monitor Response1 TRODS TRODS TRODS SYN/ACK IP IP IP ID ISN Load Balancer Client SYN ACK Request KV Store Server Server Server 25

  23. Failure Walkthrough Service TCP TCP Liveness Monitor TRODS TRODS IP IP ID ISN ! Response2 Load Balancer Response3 Client Response4 FIN ? ACK ACK ACK FIN ACK KV Store Server Server 26

  24. Related Work • New Transport • Trickles, SCTP, TCP Migrate, … • TCP • FT-TCP, ST-TCP, Backdoors, … • HTTP • CoRAL, …

  25. Implementation • Linux Kernel Module • 3,000 lines of C • ~CoRAL • Optimistic subset of CoRAL

  26. Experiments • Additional Latency • Normal • Failure • Throughput • Lighttpd @ Princeton • Apache @ Emulab • Hybrid TS & KV Throughput • Failure

  27. Normal Case Latency • TRODS-TimeStamp (TS) • Median: + 0.009 ms • 99th: + 0.012 ms • TRODS-Key-Value (KV) • Median: + 0.137 ms • 99th: + 0.148 ms

  28. Recovery Latency ~15% ~35% ~50% Blink of an eye

  29. ThroughPut Per Server Raw 120 ops/s 120 ops/s 30 ops/s/server 30 ops/s Frontend TPPS 20 ops/s/server 30 ops/s

  30. Lighttpd 9% 7% 66% 38% KV/Server: 1/8 KV/Server: 1/4 KV/Server: 1/2 KV/Server: 1/34

  31. Apache

  32. Summary • Recover Object Delivery Connections • Exploit TCP Specification to Coerce Clients • To send recovery-starting packets • To provide persistent storage • Evaluation • Low Latency • High Throughput Per Server Unmodified Unmodified ^ ^

  33. Summary • Recover Object Delivery Connections • Exploit TCP Specification to Coerce Clients • To send recovery-starting packets • To provide persistent storage • Evaluation • Low Latency • High Throughput Per Server Unmodified ^ • Questions?

More Related