1 / 20

CS162 Section

CS162 Section. Lecture 11. Project 4. Implement a d istributed key-value store that uses Two-Phase Commit for atomic operations, R eplication for performance and fault-tolerance, Encryption for security. Distributed Key-Value Store. 0. Client 1. 2 16. Client 2. 2 111. Coordinator.

aelwen
Download Presentation

CS162 Section

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS162 Section Lecture 11

  2. Project 4 • Implement a distributed key-value store that uses • Two-Phase Commit for atomic operations, • Replication for performance and fault-tolerance, • Encryption for security

  3. Distributed Key-Value Store 0 Client 1 216 Client 2 2111 Coordinator Client 3 Client 4 270 247 Client 5

  4. Consistent Hashing • SlaveServer 128-bit ID using java.util.GUID • Slave count known in advance • Nodes will come back if fails (no permanent failure) • Hash Keys to 128-bit numbers 0 27 2111 + 1 216 2111 270 247 266

  5. Client-side Program Client-side Program Client-side Program KVMessage KVClient (Library) KVClient (Library) KVClient (Library) Socket Server KVClientHandler KVMessage TPCMaster (Library) KVInterface KVCache(LRU Cache) Update Job Queue Available Threads GET PUT DEL Thread Pool Coordinator Server TPCMessage

  6. TPCMessage Key-Value Server Key-Value Server Key-Value Server Socket Server TPCClientHandler KVInterface KVInterface KVCache(LRU Cache) KVStore Job Queue TPCLog Available Threads KeyValue Server Thread Pool

  7. KVStore Client 2 Key Server KVCache Client 1 PUT (K,V) 1. Get WriteLock Update Value GET (K) 1. Get WriteLock 2. Update Value 3. Get Exclusive Lock on AccessList 4. Release WriteLock 5. Update AccessList 6. Release lock on AccessList 1. Get ReadLock Get ReadLockblocked Success 2. Release WriteLock 1. Got ReadLock 1. Get ReadLock 2. Get Value 3. Get Exclusive Lock on AccessList 4. Release ReadLock 5. Update AccessList 6. Release lock on AccessList K, V 2. Release ReadLock

  8. Client 2 Coordinator KVCache Client 1 SlaveServer 1 SlaveServer 2 PUT (K,V) 1. Get WriteLock GET (K) PREPARE RESPONSE (READY/ABORT) Phase 3 Phase 3 DECISION (COMMIT/ABORT) ACK 2PC Get ReadLockblocked

  9. Client 2 Coordinator KVCache Client 1 SlaveServer 1 SlaveServer 2 Get ReadLockblocked 1. Got WriteLock 2. Update Value 3. Get Exclusive Lock on AccessList 4. Release WriteLock 5. Update AccessList 6. Release lock on AccessList Success 2. Release WriteLock 1. Get ReadLock 1. Got ReadLock 2. Get Value 3. Get Exclusive Lock on AccessList 4. Release ReadLock 5. Update AccessList 6. Release lock on AccessList K, V 2. Release ReadLock

  10. Recovery from Site Log Entries

  11. What is MTBF?What is MTTR?

  12. Fault Models • Failures are independent*So, single fault tolerance is a big win • Hardware fails fast (blue-screen, panic, …) • Software fails-fast (or stops responding/hangs) • Software often repaired by reboot: • Heisenbugs – Works On Retry • (Bohrbugs – Faults Again On Retry) • Operations tasks: major source of outage • Utility operations – UPS/generator maintenance • Software upgrades, configuration changes

  13. Fault Tolerance Techniques • Fail fast modules: work or stop • Spare modules: yield instant repair time • Process/Server pairs: Mask HW and SW faults • Transactions: yields ACID semantics (simple fault model)

  14. Fault-tolerance in MapReduce • What if • A task crashes • A node crashes • Worker node • Master node • A task is going slow

  15. Key Security Properties • Authentication • Data integrity • Confidentiality • Non-repudiation

  16. LxvnkbgzVhffngbvtmbhgpbmaVkrimhzktiar • Vhffngbvtmbhgbgikxlxgvxhytwoxkltkbxl • Maxkxbl t dxr, ihllxllbhghypabvateehplwxvhwbgz, unmpbmahnmpabvawxvhwbgzblbgyxtlbuex • Manl, dxrfnlmuxdximlxvkxmtgwghmznxlltuex

  17. Securing Communication with Cryptography • Communication in presence of adversaries • There is a key, possession of which allows decoding, but without which decoding is infeasible • Thus, key must be kept secret and not guessable

  18. m Plaintext (m) Internet Encrypt with secret key Decrypt with secret key Ciphertext Using Symmetric Keys • Same key for encryption and decryption • Transferring keys over the network is problematic

  19. Plaintext Plaintext Internet Encrypt with public key Decrypt with private key Ciphertext Public Key / Asymmetric Encryption • Sender uses receiver’s public key • Advertised to everyone • Receiver uses complementary private key • Must be kept secret

  20. Properties of RSA • Requires generating large, random prime numbers • Algorithms exist for quickly finding these (probabilistic!) • Requires exponentiation of very large numbers • Again, fairly fast algorithms exist • Overall, much slower than symmetric key crypto • One general strategy: use public key crypto to exchange a (short) symmetric session key • Use that key then with AES or such • How difficult is recovering d, the private key? • Equivalent to finding prime factors of a large number • Many have tried - believed to be very hard (= brute force only) • (Though quantum computers can do so in polynomial time!)

More Related