1 / 15

EnsemBlue : Integrating Distributed Storage and Consumer Electronics (edited slides)

EnsemBlue : Integrating Distributed Storage and Consumer Electronics (edited slides). Consumer Electronic Devices (CEDs). Personal digital data Explosion Very different goals than: GFS, Palimpsest, xFS , OceanStore Differences? Palimpsest – no human GFS – large writes

lala
Download Presentation

EnsemBlue : Integrating Distributed Storage and Consumer Electronics (edited slides)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EnsemBlue: Integrating Distributed Storage and Consumer Electronics (edited slides)

  2. Consumer Electronic Devices (CEDs) • Personal digital data • Explosion • Very different goals than: • GFS, Palimpsest, xFS, OceanStore • Differences? • Palimpsest – no human • GFS – large writes • xFS – GP, but similarities • OceanStore – everyones data

  3. Why are CEDs different from general purpose machines? • Narrowness of interface • Degree of specialization • Computationally weak Why is it complex to manage personal digital data? • Unique data formats • Data organization schemes • Limited computing resources • Consistency • Number of files, incl. replicas • Manual movement of data

  4. Distributed Storage Solutions • DFS can help manage CEDs and multimedia My data • Based on BlueFS: • Single namespace • Supports mobile clients • Designed for small group of users • BlueFS + Ensemble = EnsemBlue • Persistent queries • Explicit support for closed-platform CED’s • Local data exchange • Disconnected operation

  5. Integrating CEDs: Leveraging general-purpose computers Distributed File System DFS protocol DFS protocol DFS protocol General-purpose client Device-specific protocol

  6. The EnsemBlue Daemon: Wolverine EnsemBlue DFS Modification of data in DFS namespace Detect modifications Update modifications Modification of data in CED namespace Wolverine Auth. copies • Runs on the general-purpose client • Acts on behalf of CED for all EnsemBlueactivities • Holds receipts for all objects on device; server callbacks

  7. Integrating CEDs (receipts) • One-to-one mapping : Namespace diversity Mapping Fully-qualified pathname of file in local CED namespace: /iPod/Songs/LetItBe.mp3 Unique EnsemBlue Identifier Object 1.999.18A • “Like” a symbolic link • File-system independent

  8. Persistent Queries • Event-notification mechanism • Reuse existing cache consistency of DFS: • Persistent query = file system object • Event notification = modification to DFS objects • Functional examples: • Transcoder (m4a => mp3) • Type-specific affinity (all jpegs => specific directory)

  9. Persistent Queries : Example (M4A to MP3 Transcoder) File Server Sets callback with the file server Client adds a new M4A file pq_ create (..) Append event record to query Application (transcodes M4A music to MP3 format) M4A Player Creates corresponding MP3 file MP3 Player

  10. More on Persistent Queries • Client-side Evaluation: Adv. to server side? • Better computational resource • Close-to-data computation (primary replicas lie on Server) • Persistent query issues • Garbage collect old queries • Overhead

  11. Disconnected devices • CEDs are often disconnected • P2P => mobility • central server => safety, consistency • Ensembles • middle-ground between P2P systems and centralized file servers • collection of devices sharing a common “view” of file system

  12. Ensembles • Ensembles store “cache” of file objects • Pseudo file server – ‘Castellan’ • Maintains a replica list of cache contents of all devices in the ensemble • Consistency of data, update propagation

  13. Ensemble : Operation Examine replica list Castellan Update replica list RPC(to fetch data) Hit: services the request Miss: sends an error code Fetches the object Client 1

  14. Conclusion • Focus on storage: CED’s integrated with DFS • User-specific “views” of FS – Ensembles • Namespace diversity • Persistent queries, disconnected operation on mobile devices

  15. Discussion • Why is consistency actually an issue? • write once, read many? • personal data: human-access, eventual consistency probably ok • Castellan is a GP computer and devices are disconnected … why isn’t Castellan connected? • How does CED invoke to Castellan • Battery power? • Other thoughts? RPC(to fetch data)

More Related