AFS

1. AFS Per-File ACLs Marc Dionne TechnoConseil

2. Outline • Introduction • History • Issues -Protocol and semantics • Issue - Implementation • Issues - Compatibility • Current status • What's next

3. Introduction • AFS user/administrator/developer for 15 years • Mainly involved in development over the past few years • Linux client • kernel updates, bugs, some improvements • Code cleanup efforts • Looking for a more substantial and challenging contribution

4. Introduction • AFS only allows ACLs to be set at the directory level • All files share this ACL • Disadvantage compared to other local or network filesystems • Users new to AFS expect ACLs to work on files • Need workarounds for common situations: dot files, .mailrc, etc • Some files need to be in a common location, but need different rights • Some demand for this feature in the community

5. History • Had questions about the feasibility – in particular the impact on the client side • Coded an initial prototype – june 2009 – over a weekend • Simplest implementation possible • New special file for file ACLs, parallels small vnode index – 1 slot per file • Existing ACL RPCs • Encouraging - surprisingly stable and functional • Very few changes needed on the Unix client • ... but some issues lurking • Discussions on afs3-std and other venues • Several revisions since then

6. Issues - Protocol and semantics • Current AFS protocol does not specify ACL operations on files • Requires new RPCs: fetchACL and storeACL • Inheritance semantics need to be defined • Are ACLs inherited, when, how • Inherit until set • Inherit always • How does an ACL change on a directory affect files • Aim for least surprise for current AFS users • Behaviour of client tools • fs setacl, listacl • vos move, restore, etc.

7. Issues - Compatibility • General • All combinations of current and new clients and servers should interoperate reasonably • OK to restrict new functionality – limit access to files that have an ACL with broader access than the parent • But not OK to expose files that file ACLs should make unreadable • Servers and clients need to be aware of the other side’s status • Use client and file server capability bits in OpenAFS • Capabilities exchange for Unix client recently merged

8. Issues - Compatibility • Current clients can leak cached data • They assume directory rights apply to all files, but rights may now vary per file • Scenario • Users A and B can read directory D • File F has a file ACL that allows A to read, but not B • A reads F, data is brought into the cache • B tries to read F, cache manager assumes rights to D apply, and happily returns data • Possible solution: artificially restrict rights • Fiddle with the returned user rights on the server side, or the file mode bits • Compute most restrictive rights for that user for all files within a directory – return the same rights for all files

9. Issues - Compatibility • Windows • Tests showed the Windows client reacted badly with files more restricted than the parent directory • Mainly lengthy hangs in explorer • Commit 9e8ae43b introduced a registry key to correct this behaviour • Should be activated based on server capabilities • Same solution should apply – return most restrictive rights in the directory • Volume moves and restores • Prevent data (ACLs) loss while moving volumes to an older server • Allow forced moves

10. Issues - Implementation • Changes are required to the existing on-disk structures • On-disk Vnode structure is full • RXOSD already repurposes some elements (vnode “magic”) • Really need a pointer in the Vnode – alternatives are much more complex (hashing, etc.) • Current scheme requires a power of 2 size • Small vnode size would have to double • May be a concern for sites with large numbers of files • Volume header is nearly full • RXOSD repurposes an existing file pointer • New volume data (file ACLs) need to be preserved across volume clone, dump, restore and move operations • New dump tags

11. Current status • Prototype implentation • Published as a github clone:http://github.com/mdionne/openafs • New per volume special file for file ACLs • Reference counted entries, although 1 entry per file currently • Linked entries to track available slots • Reuse “magic” field in on-disk vnode structure as an ACL pointer • known conflict with RXOSD • In memory, file ACL follows the on-disk vnode structure (similar to directories) • ACL is stored and read along with the Vnode (VnLoad, VnStore) • ACL modification triggers vnode writeback

12. Current Status • New fetchACL and storeACL RPCs defined and used • Identical signature to current RPCs • New clearACL RPC needed • Client capability identifies file ACL support • Used to determine whether fileserver should restrict rights • Rights restrictions not implemented yet • Some security concerns whether it's acceptable to rely on capabilities to trigger this • Server capability • Clients know not to assume that directory rights apply to all files • Clients use new ACL RPCs

13. Current Status • Inherit until set semantics • Once a file ACL is set, it is “detached” from the parent - ACL changes to the parent will no longer affect it • New files have no file ACL – parent ACL applies • fetchACL returns a special value to indicate no ACL • New clearACL RPC to re-attach to parent ACL • listacl • Show file ACL, or an indication that there is none • Option to show effective ACL • Volume dumps • New tag identifies a file ACL • ACL retrieved from special file and added to dump if needed • On restore, insert ACL into target volume special file

14. What's Next • Getting consensus and document protocol changes and semantics (afs3-std) • RPC refresh – new ACL RPCs • Consensus on on-disk structures and implementation, particularly the possible Vnode expansion • Unimplemented features • Restrict rights for legacy clients • Windows client changes • Volume manipulation safeguards • Documentation changes • Goal: keep the scope under control • Better chances of getting it done and integrated in a reasonable timeframe

15. Parting thoughts • Code is easy - getting consensus is harder • Small number of key people • Few opinions, some disagreement • File server code is more intuitive than cache manager code • And userspace code is easier to debug than a kernel module • But bugs can be more painful... • Dependencies and links with other pending work don't help • RPC refresh, out of tree projects (RXOSD) • Other ongoing or potential projects: alternate data streams, extended attributes, etc.

16. Thank you Questions or comments ?