1 / 24

Overview of Lustre

Overview of Lustre. ECE, U of MN Changjin Hong (Prof. Tewfik ’ s group) hongcj92@ece.umn.edu Monday, Aug. 19, 2002. Outline. Reference Lustre Cluster Lustre System Components Distributed Lock Manager Object Based Storage Conclusion (security issues). Reference.

dewei
Download Presentation

Overview of Lustre

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of Lustre ECE, U of MN Changjin Hong (Prof. Tewfik’s group) hongcj92@ece.umn.edu Monday, Aug. 19, 2002

  2. Outline • Reference • Lustre Cluster • Lustre System Components • Distributed Lock Manager • Object Based Storage • Conclusion (security issues)

  3. Reference • Lustre: A SAN File System for Linux • http://www/lustre.org/docs/lustre/luswhite.pdf • Several presentation materials from Dr. Peter J. Braam

  4. A Lustre Cluster 10,000’s 10’s of nodes 1,000’s

  5. Key Design Issue : Scalability • I/O throughput • How to avoid bottlenecks • Metadata scalability • How can 10,000’s of nodes work on files in same folder • Cluster Recovery • If sth fails, how can transparent recovery happen • Management • Adding, removing, replacing, systems; data migration & backup

  6. System Components

  7. Interaction between systems Pre-allocation file creation, recovery purpose, file status, MDS CMD protocol (directory) metadata handling, inodes updates, concurrency Client OS protocol File I/O, allocation of blocks, striping, security enforcement OST

  8. Client File System • A directory tree, subdivision into filesets for cluster ▷wide Unix file sharing semantics • CMD protocol • Transaction-based • Authenticated access • Write-behind caching for MD updates with strict data/metadata coherency

  9. Metadata Service (MDS) • All access to the file is governed by MDS which will directly or indirectly authorize access. • To control namespace and manage inodes • Load balanced cluster service for the scalability (a well balanced API, a stackable framework for logical MDS, replicated MDS) • Journaled batched metadata updates

  10. Object Storage Targets (OST) • Keep file data objects • File I/O service ▷Access to the objects • The block allocation for data obj., leading distributed and scalability • OST s/w modules • OBD server, Lock server • Obj. storage driver, OBD filter • Portal API

  11. VAXCluster DLM adapted

  12. Distributed Lock Manager • For generic and rich lock service • Lock resources: resource database • Organize resources in trees • High performance • node that acquires resource manages tree

  13. Big Picture Resource Tree and namespace Resource manager Obj.1 <namespace> Name1 Name2 Name3 Name4 : R R Obj.2 R R Obj.3 distributed resource directory/hash function (LDWV)/lock directory Obj.4 Apps.

  14. Mechanism in resource dB • Hash binary string % N ▷ get h • Lookup system in lock directory weight vector [h] ▷ find system K. • Systems • may occupy 0, 1 or more slots in LDWV • Number of slots is lock directory weight

  15. Lustre DLM features • Low concurrency • Want write-back caching • High concurrency • Want load balancing in cluster • Subdivide directories etc with hashes • Want server of request to limit lock revocations-> ops. on the MD cluster in a client server RPC model • Deadlock detection

  16. Object Based Storage

  17. Object Based Storage • Object Based Storage Device • More intelligent than block device • Speak storage at “inode level” • create, unlink, read, write, getattr, setattr… • Iterators, security, almost arbitrary processing

  18. Components of OB Storage • Storage Object Device Drivers • Class drivers : attach driver to interface • Targets, clients : remote access • Direct drivers : to manage physical storage • Logical drivers: for intelligence & storage management • Object storage application (OSA) • (cluster) file systems • Advanced storage : parallel I/O, snapshots • Specialized apps. : caches, db’s, filesrv

  19. System Interface • Modules • Load the kernel modules to get drivers of a certain type • Name devices to be of a certain type • Build stacks of devices with assigned types

  20. Layering of Object Drivers

  21. Interaction of Obj. Storages/w modules

  22. Benefits-clustering/SM • Suitable for use in a SAN file system • Shared at the level of an individual block • Obj namespace : divided into obj group. This is very advantageous to be able to create obj w/ given obj id’s. Good for snapshot! • Hot file migration

  23. Conclusion • Object Based Storage To process the disk operations on the higher concept of individual files and the file inode level, rather than the low-level h/w disk block level. • Security Issues • Auxiliary service in cluster • LDAP, PKI, Kerberos • Purpose • CFS/ MDS/ OST • Authenticate to each other • Set up session keys

  24. Etc. • GSS-API for authentication and Integrity Checks • Remote DMA • Layer for NEVER bypass security processing • Request processing for checking authentication by a higher level layer in the networking stack

More Related