1 / 15

ISTORE: An Introspective Storage Architecture for Network Service Applications

ISTORE: An Introspective Storage Architecture for Network Service Applications. Aaron Brown, David Oppenheimer, Kimberly Keeton, Randi Thomas, Jim Beck, John Kubiatowicz, and David Patterson http://iram.cs.berkeley.edu/istore 1999 Winter IRAM Retreat. Agenda.

william-walls
Download Presentation

ISTORE: An Introspective Storage Architecture for Network Service Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ISTORE: An Introspective Storage Architecture for Network Service Applications Aaron Brown, David Oppenheimer, Kimberly Keeton, Randi Thomas, Jim Beck, John Kubiatowicz, and David Patterson http://iram.cs.berkeley.edu/istore 1999 Winter IRAM Retreat

  2. Agenda • Overview of ISTORE: Motivation and Architecture • Hardware Details and Prototype Plans • Software Architecture • Discussion and Feedback

  3. Motivation • Emerging paradigm of network-based services • today: e-commerce, online database services, search engines, and web servers • tomorrow: above, plus thin-client/PDA infrastructure support • These services have unique demands • vast storage needs (large operational and historical DSS databases) • high access rates • mix of transactional (ACID) and non-transactional accesses • easy incremental scaling of storage and processing • minimal administration burden • easy setup, scaling, maintenance, repair, tuning, ... • automatic dynamic adaptation to failures, workload changes, ...

  4. Scaleable Storage Appliances • Best architecture to meet these demands is a scaleable self-maintaining storage appliance • appliance:a single-function device optimized to provide one application or service • consists of software and hardware constructed with that service in mind • interface to WAN/LAN via standard high-level protocols • internal implementation hidden • storage appliance:integrates an application with its data storage in one unit • scaleable appliance:incrementally scaleable with automatic integration of new resources (e.g., disks) • self-maintaining appliance:runs autonomously, without human intervention, once initially configured • Examples: e-commerce, decision-support DB, PDA proxy, video server, or online voting appliances

  5. Today’s Appliances • Some storage appliances exist today: • Network Appliance NFS and proxy servers • SNAP! Server file server • Qube web server • SonicWall firewall • ... • But none are truly self-maintaining, and most are not scaleable

  6. Why Aren’t There Appliances That Match Our Criteria? 1) Self-maintenance is tricky and application-dependent • can’t rely on human administrators to configure, monitor, and tune system for its application • failure management • devices must fail fast without interrupting service • failures should not require immediate human intervention • performance management • system must adapt to changes in workload or access patterns • system upgrades and scaling • new hardware resources must be automatically incorporated without interruption of service • new devices should be used immediately to improve performance or repair prior failures

  7. More Appliance Challenges 2) Appliance designs are inextricably tied to single applications • each service or application requires a custom-designed appliance • new systems software must be written, tested, debugged • often can share hardware, but entire system must be retested • especially difficult to design new self-maintaining appliances • performance and manageability suffer when an appliance is used for other than its intended purpose • e.g., if workload doesn't quite match appliance design goals • example: using a file-server appliance as the storage subsystem for a database application

  8. The ISTORE Solution:An Introspective Meta-Appliance • A meta-appliance is a generic hardware and software platform that can be specialized into a single-application, focused appliance • An introspective system uses hardware and software monitoring to tune itself and to achieve self-maintenance • The ISTORE meta-appliance is a hardware/software architecture for building scaleable self-maintaining storage appliances

  9. ISTORE Hardware Innovations 1) ISTORE uses intelligent hardware • ISTORE is a shared-nothing distributed system of intelligent device bricks interconnected by an intelligent chassis CPU, memory, NI IntelligentChassis Device IntelligentDevice Brick • intelligent device brick: a device (e.g., a disk) plus a fast embedded CPU, memory, and NI (e.g., an IRAM) • intelligent chassis: scaleable, redundant, fast network + UPS

  10. Benefits of ISTORE’s Hardware Architecture • Decentralized processing (shared-nothing) • system can withstand partial failure • Intelligent devices monitor their own “health,” test themselves, manage failures, collect application-specified performance data, and execute application worker code • provides the foundation for self-maintenance and self-tuning • Plug & play, hot-swappable bricks ease configuration, scaling • hardware is specialized/scaled by selecting an appropriate collection of devices • disks, DRAMs, WAN/LAN interfaces, ...

  11. ISTORE Software Innovations 2) ISTORE’s system software is designed to deliver self-maintenance and allow for easy specialization • layered mechanism libraries facilitate introspection, self-maintenance • extensibility via domain-specific languages (DSLs) allows customization of the runtime system with application-specific policies • Benefits of ISTORE’s software architecture • appliance designer inherits functionality of supplied base mechanisms for monitoring and adaptation • easy for designer to customize mechanisms and define adaptation policies by writing domain-specific code

  12. ISTORE and IRAM • ISTORE relies on intelligent devices • IRAM is an easy way to add intelligence to a device • embedded, low-power CPU meets size and power constraints • integrated DRAM reduces chip count • fast network interface (serial lines) meets connectivity needs • Initial ISTORE prototype won’t use IRAM • will use collection of commodity components that approximate IRAM functionality, not size/power • but results will be applicable to IRAM-based systems • success with ISTORE will encourage integration of IRAM-like processors into standard devices

  13. Agenda • Overview of ISTORE: Motivation and Architecture • Hardware Details and Prototype Plans • Software Runtime Architecture • Discussion and Feedback

  14. Backup Slides

  15. Related Work • ISTORE extends several recent research efforts • Intelligent/Active Disks, NASD (UCB, UCSB, CMU) • used as a component in ISTORE; ISTORE adds a global runtime system, introspection, and self-maintenance • network service appliances (NetApp, Snap!, Qube, Swarm, ...) • ISTORE adds the idea of a meta-appliance & self-maintenance • self-maintaining systems (Sun Project StoreX, Tandem, ...) • extensible operating systems (SPIN, Exokernel, VINO, Scout,...) • domain-specific languages (OGI Microlanguages, Prolac, VHDL,...) • adaptive systems (HP AutoRAID, MS AutoAdmin, MS Millennium) • plug-and-play system construction (Sun Jini, PC Plug&Play, ...) • ISTORE leverages Berkeley’s experience with reliable, large, distributed storage systems and applications • RAID, Tertiary Disk, XFS, Sprite, NOW, PostGres/GiST, ...

More Related