Introspective Replica Management Yan Chen, Hakim Weatherspoon, and Dennis Geels

1. Introspective Replica ManagementYan Chen, Hakim Weatherspoon, and Dennis Geels • Our project developed and evaluated a replica management algorithm suitable for an environment such as OceanStore, which is: • Federated: servers make autonomous decisions. • Insecure: servers, infrastructure distrusted, may be adversarial. • Read/Write: replicate data may be updated, must remain consistent. • Extremely Wide Area: algorithm must be scalable and automated. • Heterogeneous: servers vary in performance and capacity.

2. & Related Work Motivation • RaDaR, from AT&T, explored introspection in a trusted, homogeneous, read-only WAN environment. • Mariposa, from U.C.Berkeley, assumed trusted but autonomous servers, optimizing through an economic model. • DBMS caching projects have optimized replication within single administrative domains and on a local scale (such as ADR algorithm by Wolfson et al). • Replica Management has a large impact on system performance. • Too many replicas increase overhead of commit protocol. • Too few replicas cannot handle load of clients’ requests. • Poor placement increases client-server communication latency. • OceanStore requires secure algorithm suitable for extremely large scale. • The size of the system prevents manual optimization. • An Introspective system learns workload patterns and adjusts automatically.

3. Floating replica GUID Version Size Peers AccessRecords Data d1 High-speed interconnect d3 d1 d1 d2 d2 d3 floating replicas System Model d1 d5 Naming & Location servers d2 d1 d3 d2 d2 Preference Path clients • Clients access data through a nearby replica which is not overloaded. • The degree of replication and replica location are automatically • adjusted to decrease end-to-end latency as seen by the client.

4. Design • Distributed • Introspection. • Load balancing by autonomy: Servers make decisions based on local knowledge. • Stability • Load stability: high watermark and low watermark. • Performance stability: a conservative approach is taken when making introspective decision. • Security • Authenticated, signed requests from the client. • Quorum decision for deletion and grant of privileges to floating replica. • Gratuitous migration/replication masked through hysteresis and suicide.

5. Algorithm • Floating Replicas service Client requests. • Read requests are serviced by a nearby replica which is not overloaded. • Updates require agreement by a quorum of replicas (Byzantine fault tolerant protocol by Castro & Liscov). • Observation component learns workload patterns. • Replicas record distance to clients, number of requests. • Servers extract network topology from routers and message headers, meter resource consumption. • Optimization component tunes system for better performance. • Replicas are created, moved, or deleted, to minimize request latency. • Servers free to decide they are overloaded and to refuse new work.

6. Simulation • Simulation Environment. • Used web proxy trace from DEC as workload. • Created a network topology by GT-ITM and placed servers randomly. • GT-ITM is a topology generator from Georgia Tech. • All systems were developed in Java with the gcj compiler. • Single process trace-driven simulation. • Explored and validated potential of our algorithm. • Distributed Network Simulator. • Distributed real Servers and Floating Replicas across Millennium cluster. • Our Network Simulater provided a WAN simulation. • delayed and routed messages appropiately. • collected network statistics. • Reliable UDP layer for efficient communication. • More of a realistic evaluation of scheme.

7. Results

8. Results, cont…

9. d2 d3 Performance Gain over RaDaR • Extend preference path for write • Consider migration and replication simultaneously High-speed interconnect d3 d1 d5 Naming & Location d1 d1 d1 d2 Preference Path d1 d1 clients

10. Conclusions • Designed a federated, secure, read/write introspective replica management system for heterogeneous wide-area environment. • Preliminary implementation and testing shows reduced E2E latency, network bandwidth resource consumption, and storage (by removing gratuitous floating replicas). Also, we achieve distributed load balancing as well. • Future Work • Incorporate distributed naming and location mechanism. • I.e. through bloom filter. • Automatic tuning of the introspection parameters. • Implement authentication for client requests and quorum decisions for deletion/creation of floating replicas.