1 / 8

Reliable Adaptive Distributed Systems

Explore the world of reliable, adaptive distributed systems with a focus on statistical learning theory, verification methods, and network services. Discover how cutting-edge technologies are revolutionizing system dependability, security, and error handling for improved performance.

bguidry
Download Presentation

Reliable Adaptive Distributed Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reliable Adaptive Distributed Systems Armando Fox, Michael Jordan, Randy H. Katz, David Patterson, George Necula, Ion Stoica, Doug Tygar

  2. Motivations and Observations • Existing systems: • Fragile, easily broken, poor reliability/security • Overwhelming configuration complexity, infrequently correctly performed, yielding lack of dependability, significant vulnerabilities • Magic bullets? • Statistical learning theory: foundation for algorithms that observe/predict future behaviors • Verification technology—check for correct behavior, reveal vulnerabilities, automatically generate behaviors with desirable properties • Programmable network elements—“active” code inserted into network, provide observation/enforcement points without access to user end systems

  3. The Team and Approach • Team: • Statistical Learning Theory (SLT) (Michael Jordan) • Network Services/Protocols (Armando Fox, Randy Katz, David Patterson, Ion Stoica) • Verification Methods for program, network, security behaviors (George Necula, Ion Stoica, Doug Tygar). • Approach: comprehensive distributed system architecture • SLT building block/practical components for distributed system observation, coordination, inference, correction, and evolution of behaviors • Network behaviors and how they reveal operation of higher-level network applications • Key enabler: embed observational and inference means at strategic points in the network, avoiding modification of end hosts or apps (aka “knowledge plane”) • Apps: web services, intrusion detection, storage access, etc. • Improvements: • Dependability enhanced by monitoring network state, rapidly detecting behavioral changes (e.g., failures), configuring new resources in response • Security enhanced thru more rapid discovery of and response to attacks

  4. Block the in-coming attack Contain outgoing attacks Secure the edge network Patch here once Network is patched Block things rather thanupgrade software! Not the end systems Network is smart Not the end systems Fragile to change, upgrade Old System Old System Old System Redundancy Isolation/Containment Heterogeneity Old System Too difficult to apply patches here

  5. Make-A-Difference Technologies • Statistical Learning Theory (Michael Jordan) • Toolbox for the design and analysis of adaptive systems • New and scaled-up algorithms for classification, diagnosis, prediction, novelty detection, outlier detection, quantile estimation, density estimation, feature selection, variable selection, response surface optimization, sequential decision-making • kernel machine: functional analysis plus convex optimization, yielding generalized inner product to measure similarities among data point pairs • novelty detection/quantile estimation problems: given cloud of data in feature space, place boundary so as to guarantee only small fraction falls outside (second-order cone program—convex optimization with efficient solution methods) • Challenge: make these algorithms work “on-line” while embedding them within network and distributed systems architectures

  6. Make-A-DifferenceTechnologies • New Approach to Run-Time Error Handling (George Necula) • Program-level error-handling is difficult • Repetitive, hard to maintain and specify • Standard run-time approach is abort or ignore! • Necula’s approach • Add explicit support at the programming language level • Make compensations and interface obligations • First-class citizens (like objects) • With Static and Dynamic checking • Handles tricky features (e.g., loops, the heap) • Still provides strong guarantees • Like sagas or compensating transactions • Standard transactions not appropriate for this use

  7. Buffers Buffers Buffers Input Ports Output Ports CP CP CP CP CP CP AP CP Interconnection Fabric Action Processor Classification Processor Implementation Platform based on Programmable Networks Tag Mem Rules & Programs

  8. Short Statements • Patterson/Fox: Dependability of Distributed Systems • Stoica: Trust in Distributed Systems through Protocol Verification • Tygar: Security in Distributed Systems

More Related