1 / 45

P1 : Distributed Bitcoin Miner

This document provides debugging tips for the Distributed Bitcoin Miner project, including logging techniques, understanding tests, and implementing the Live Sequence Protocol.

vernonmoore
Download Presentation

P1 : Distributed Bitcoin Miner

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. P1 : Distributed Bitcoin Miner 15-440/15-640 09/17/2019

  2. Debugging Tips P0 Solution P1 Part A Overview

  3. Debugging Tips • Logging • Add log statements around points of inter-thread communications • Channel • Connection • … • Attach server/client IDs to each log line • Create multiple log files, one for each thread • Remember to run with –race flag and read –race output • Goland has a built-in debugger • (Demo)

  4. Debugging Tips • Logging • Limit log output size for easy navigation • Convenient to use flags to enable/disable logging • No multi-level logging support in Go’s standard library • Beware of the complaints from ‘-race’ when a single logger is used

  5. Debugging Tips • Why read the tests programs? • Understand the expected behavior of the program. • The system specification is written in natural language and thus inherently ambiguous and prone to misinterpretation. • The test program is more precise. • Prepare for Part B. • The public tests are simple. The hidden tests are brutal. • You will need to write some of your own tests for partB!

  6. P0 Reference • Reference solution to p0 from a previous semester will be posted • Should be structurally identical

  7. Part A Checkpoint (Due 9/24) Part A (Due 10/5) Part B (Due 10/15) Timeline

  8. Part A: LSP Protocol • You will implement the Live Sequence Protocol • LSPis similar to TCP, it adds functionality to UDP • LSP has some of its own features

  9. LSP Features • LSP supports its own client-server communication model • Server communicates with multiple clients • Received messages must be processed in order • LSP includes Sliding Window Protocol • Payload size and Checksum are used to verify data integrity. • LSP includes Epoch Events for Retransmission and Timeout Mechanism

  10. Messages • Each message is consists of: • Message Type: Connect, Data, Ack • Connection ID: uniquely identifies each client-server connection • Sequence Number: sequence number increments with each message sent • Payload Size: used to verify data integrity • Checksum: used to verify data integrity • Payload

  11. Messages type MsgType int const ( MsgConnect MsgType = iota // Conn request from client. MsgData // Data message from client or server. MsgAck // Acknowledgment from client or server. ) type Message struct { Type MsgType // One of the message types listed above. ConnID int // Unique client-server connection ID. SeqNum int // Message sequence number. Size int // Size of the payload. Checksum uint16 // Checksum of the message. Payload []byte // Data message payload. }

  12. Messages • Message size is limited to single UDP-packet size • Each Message is received exactly once (ignore duplicates) • Messages are marshaled using Go’s Marshalfunction in the json package and sent as a UDP packet

  13. Client-Server Communication:Establishing a Connection Server Client (Connect, 0, 0) Client begins by sending a connection request (must have ID 0 and sequence number 0)

  14. Client-Server Communication:Establishing a Connection Server Client (Connect, 0, 0) (Ack, id, 0) Server generates a unique connection id for this Client-Server connection (you can just generate ID’s sequentially)

  15. Client-Server Communication:Sending & Ack-ing Data Server Client (Data, id, i, “hello”) Server and Client maintain independent sequence numbers. (Ack, id, i) (Data, id, i+1, “hi”) (Ack, id, i+1) (Data, id, j, “hi”) (Ack, id, j)

  16. Received messages must be processed in order. Server UDP Packets aren’t guaranteed to arrive in order. LSPServer.Read()//blocks LSPServer.Read() LSPServer.Read()

  17. Received messages must be processed in order. Server (Data, id, i, “440”) UDP Packets aren’t guaranteed to arrive in order. LSPServer.Read()//returns “440” LSPServer.Read() LSPServer.Read()

  18. Received messages must be processed in order. Server (Data, id, i, “440”) UDP Packets aren’t guaranteed to arrive in order. (Data, id, i+2, “fun”) LSPServer.Read()//returns “440” LSPServer.Read()//blocks LSPServer.Read() i + 2 : “fun”

  19. Received messages must be processed in order. Server (Data, id, i, “440”) How to maintain order? (Data, id, i+2, “fun”) LSPServer.Read()//returns “440” LSPServer.Read()//returns “is” LSPServer.Read()//returns “fun” (Data, id, i+1, “is”) i + 2 : “fun”

  20. Sliding Window Protocol LSP uses a sliding window protocol Given a window size ω, we can send up to ω messages without acknowledgement. If the oldest unacknowledged message has sequence number n, then only messages with sequence numbers n + ω - 1 (inclusive) may be sent i.e. [n, n+ ω -1] In addition, number of unacknowledged messages cannot exceed MaxUnackedMessages • • • •

  21. Sliding Window Protocol Server Client ω = 3 Client messages queue = “h” ->“e” -> “l” -> “l” -> “o”

  22. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 Client messages queue = “e” -> “l” -> “l” -> “o” Oldest Seq # without Ack = i Window = [i, i+2]

  23. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “l” -> “o” Oldest Seq # without Ack = i Window = [i, i+2]

  24. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “o” (Data, id, i+2, “l”) Oldest Seq # without Ack = i Window = [i, i+2]

  25. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “o” (Data, id, i+2, “l”) Oldest Seq # without Ack = i block Window = [i, i+2]

  26. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “o” (Data, id, i+2, “l”) Oldest Seq # without Ack = i (Ack, id, i+1) Window = [i, i+2]

  27. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “o” (Data, id, i+2, “l”) Oldest Seq # without Ack = i (Ack, id, i+1) Window = [i, i+2] block

  28. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = “l” -> “o” (Data, id, i+2, “l”) Oldest Seq # without Ack = i+2 (Ack, id, i+1) Window = [i+2, i+4] (Ack, id, i)

  29. Sliding Window Protocol Server Client (Data, id, i, “h”) ω = 3 (Data, id, i+1, “e”) Client messages queue = (Data, id, i+2, “l”) Oldest Seq # without Ack = i+2 (Ack, id, i+1) Window = [i+2, i+4] Note: if MaxUnackedMessages = 1, then we cannot send i+3 or i+4 (Ack, id, i) (Data, id, i+3, “l”) (Data, id, i+4, “o”)

  30. Payload size and Checksum • Both payload size and checksum are used to verify data integrity. • Payload Size (What if received data is shorter? longer?) • Checksum • Carries more information than payload size • Can detect flipped bits introduced in the process of data transmission and storage • See writeup section 2.1.5 for detailed description of the 16-bit one’s complement sum algorithm

  31. Epoch Events • We need to deal with dropped packets. • Time interval between two epochs (δ) is fixed. • Clients and server take epoch actions when a periodic timer trigger fires • For every data message that isn’t acknowledged yet, resend when CurrentBackOff epochs have elapsed • CurrentBackOff increases according to exponential backoff rules, until it reaches MaxBackOff • If received nothing in the past epoch, send a Heartbeat message — keeps the connection alive

  32. Client Epoch Actions:Connect Message Retransmission If connection request has not been acknowledged, resend connection request, every epoch • Server Client (Connect, 0, 0) (Connect, 0, 0) (Ack, id, 0) }δ (Connect, 0, 0)

  33. Client Epoch Actions:Data Message Retransmission For every unacknowledged data message sent, resend the data message • Server Client } (Data, id, i, “data”) (Data, id, i+1, “dayda”) CurrentBackoff (Ack, id, i+1) (Data, id, i, “data”) (Data, id, i+1, “dayda”) Note that message i+1 is duplicated on the server

  34. Client Epoch Actions:Data Message Retransmission For every unacknowledged data message sent, resend the data message • Server Client } (Data, id, i, “data”) (Data, id, i+1, “dayda”) CurrentBackoff Which message(s) to resend on each epoch? (Ack, id, i+1) (Data, id, i, “data”) (Data, id, i+1, “dayda”) Note that message i+1 is duplicated on the server

  35. Client Epoch Actions:Data Message Retransmission For every unacknowledged data message sent, resend the data message • Server Client } (Data, id, i, “data”) (Data, id, i+1, “dayda”) CurrentBackoff Which message(s) to resend on each epoch? (Ack, id, i+1) (Data, id, i, “data”) (Data, id, i+1, “dayda”) What should the server do if it receives a repeat? Note that message i+1 is duplicated on the server

  36. Client Epoch Actions:Is the connection dead? • If the client: • has received Ack message for the Connect request; • has not received any Data message; • Then it should send an ack with sequence number 0 • (Heartbeat) • Server Client (Data, id, j, “hi”) (Ack, id, j) }δ (Ack, id, 0)

  37. Server epoch actions are very similar to client epoch actions. • For each client connection: • For each data message that has been sent, but not yet acknowledged, resend the data message • If no data message has been received from the client, then send an ack with sequence number 0 •

  38. Epoch Events: EpochLimit We can keep track of epochs passed since the last message was received. If this goes over a limit, we can assume the connection is lost.

  39. Read(), Write(), Close(), CloseConn() • We don’t have time to go thru these during • recitation, please go over the handout, Server.api.go, • and Client.api.go • Note that Close() is blocking while CloseConn() is not • Close() ensures that all pending messages to send are • sent and acknowledged before Close() returns • If Read(), Write, Close(), or CloseConn() is called after • Close() is called, they must either return an error, or • never return anything

  40. Checkpoint (due 9/24) • Assume no packet loss (no need to implement epoch) • Race conditions will not be checked • Messages might be sent out of order: • Need to receive messages in order • Need to implement Sliding Window Protocol

  41. Server Client Checkpoint (due 9/24) (Data, id, i, “hello”) (Ack, id, i) (Data, id, i+1, “hi”) Simple Read/ Write Server (Ack, id, i+1) (Data, id, j, “hi”) (Ack, id, j)

  42. Server Checkpoint (due 9/24) (Data, id, i, “440”) (Data, id, i+2, “fun”) Simple Read/ Write Server + Receiving In Order (Data, id, i+1, “is”)

  43. Server Client Checkpoint (due 9/24) (Data, id, i, “h”) (Data, id, i+1, “e”) (Data, id, i+2, “l”) Simple Read/ Write Server + Receiving In Order + Sliding Window Protocol (Ack, id, i+1) (Ack, id, i)

  44. lspnet Contains every UDP operation needed. • net package is not allowed! • import “github.com/cmu440/lspnet” addr, err := lspnet.ResolveUDPAddr("udp", hostport) udpConn, err := lspnet.ListenUDP("udp", addr) n, cliAddr, err := udpConn.ReadFromUDP(buffer[0]:) udpConn.WriteToUDP(msg, cliAddr)

  45. Implementation notes No locks and mutexes • There’s no limit on message queue size, so don’t use buffered channel to store pending messages as in p0. Instead use something like linked list. •

More Related