Enforcing High Level Protocols in Low-Level Software

1. Enforcing High Level Protocols in Low-Level Software Robert DeLine and Manuel Fahndrich

2. Vault • C like language, with references • Adds “capabilities” to the language • Capabilities are a static notion, so are only used at type-checking time, and carry no runtime overhead

3. Typing environment • Includes standard  for binding names to types • Additionally, we include a set C to determine if a capability is present • As expected, C starts out empty • However, while  will be full of bindings at the end of a function (remember it’s a C-like language) C must be empty

4. Tracked Types Start with tracked types: • Declared by tracked(K) T v = e • T is an arbitrary type, but not a tracked type • e is an expression of type tracked(K) T • K is a name for the key associated with the object returned by e • The key bound to K is also in the capability set, indicating that v is accessible • Keys are created, and added to capability set C, by new tracked … • Because keys are created only by new tracked, and are part of the type, the type gains a one-to-one mapping to the lifetime of each object. Any attempt to assign a different instance to the variable will fail, since it cannot have the same key, let alone lexical key. • Keys are removed by free(v) where the key is read out of the type for v

5. Example void foo() { tracked(K) point p = new tracked point {x = 1; y = 2;} p.x = 2; p.x *= 2; … free(p); } C starts out empty. After point p is created, C contains a new key, named by K. Key K persists until the call to free, after which point, key K is no longer in the capability set Reading or writing the fields of p becomes invalid once K is removed

6. Control Flow tracked(K) point p = …; bool b = random_bool(); if(b) free(v); if(!b) free(v); Vault requires that the capability sets be equal for all incoming edges at a join point. The code snippet above does not type check, because, after the first free, there is a join point, from which one edge has capability K, but the other edge does not. The capability sets do not match, so type checking fails.

7. Capabilities and functions • Currently, resources cannot escape functions. This is not useful, if we want to write a file library, for example. • This is caused by the restriction that there are no capabilities at function entry and exit. If an argument has a tracked type, it would be unusable in the function body • By annotating the function type we can relax that requirement

8. File handle example tracked(H) FILE fopen(…) [new H] void fread(tracked(H) FILE f, …) [H] void fwrite(tracked(H) FILE f, …) [H] void fclose(tracked(H) FILE f) [-H] This demonstrates how the function types are annotated. When type checking the body of the function, Vault verifies that the function meets the type spec for consuming, or creating capabilities. [new H] means that H is a fresh capability. [H] means that key H is required before function entry, and is still available upon return. [-H] removes key H from the capability set upon return. When type checking users of the code, Vault is able to check that the capabilities are used correctly. Here a capability is an open file handle. If a function does not declare that it leaves a file handle open in its type, Vault will ensure that there are no dangling open file handles for all function exit points. The function annotations do not make any sense if there is no lexical matching key elsewhere in the type.

9. Capabilities with state • Capabilities thus far are limited to binary state. Good for object lifetimes, but not much else. • To handle objects with more complicated interfaces (e.g. sockets), Vault adds state to capabilities. • Whereas before we had capability K, now we can have capability K at state s: K@s • Changing the state of a capability is handled at function boundaries only. We add arrows to the function capability annotations.

10. Socket example interface SOCKET { type sock; variant domain [‘UNIX | ‘INET]; variant comm_style [‘STREAM | ‘DGRAM]; tracked(@raw) sock socket(domain, comm_style, int); struct sockaddr {…}; void bind(tracked(S) sock, sockaddr) [S@raw->named]; void listen(tracked(S) sock, int) [S@named->listening]; tracked(N) sock accept(tracked(S) sock, sockaddr) [S@listening, new N@ready]; void receive(tracked(S) sock, byte[]) [S@ready]; void close(tracked(S) sock) [-S]; } void handle_requests() { tracked mysock socket(‘INET, ‘STREAM, 0) bind(mysock, new sockaddr{port = 80, …}); listen(mysock, 10); while(true) { sockaddr = new sockaddr(); tracked connect_sock = accept(mysock, sockaddr); …; // do actual handling code close(connect_sock); } }

11. Using sockets correctly void handle_requests() { tracked mysock socket(‘INET, ‘STREAM, 0) bind(mysock, new sockaddr{port = 80, …}); listen(mysock, 10); while(true) { sockaddr = new sockaddr(); tracked connect_sock = accept(mysock, sockaddr); byte [] buf = new buf[20]; receive(connect_sock, buf); …; // do actual handling code close(connect_sock); delete sockaddr; delete buf; } close(mysock); }

12. Receive on wrong socket void handle_requests() { tracked (A) mysock socket(‘INET, ‘STREAM, 0) bind(mysock, new sockaddr{port = 80, …}); listen(mysock, 10); while(true) { sockaddr = new sockaddr(); tracked (B) connect_sock = accept(mysock, sockaddr); byte [] buf = new buf[20]; // capability set: {A@listening, B@ready} receive(mysock, buf); // error: mysock has type “tracked(A) sock” // capability A is in listening state // receive requires the capability be in ready state …; // do actual handling code close(connect_sock); delete sockaddr; delete buf; } close(mysock); }

13. Forgetting to close socket void handle_requests() { tracked (A) mysock socket(‘INET, ‘STREAM, 0) bind(mysock, new sockaddr{port = 80, …}); listen(mysock, 10); // capability set: {A@listening} while(true) { // type error: capability sets of predecessors do not match sockaddr = new sockaddr(); tracked (B) connect_sock = accept(mysock, sockaddr); byte [] buf = new buf[20]; receive(connect_sock, buf); …; // do actual handling code //close(connect_sock); delete sockaddr; delete buf; // capability set: {A@listening, B@ready} } close(mysock); }

14. Parameterized types • Socket example is good, but doesn’t have room for error handling. What if bind fails? Is the socket in the raw state, or the named state? • Currently, we associate exactly one state with a capability, so cannot fix this • Solution: parameterized variant types

15. Parameterized types • variant option<key K> [ ‘SOME{K} | ‘NONE] • When constructing a key-parameterized type, the key is consumed by the type. • When deconstructing, the key is added back into the capability set • With this, we can force the users of code to check error codes

16. Sockets with error checking variant status<key K> [‘Ok {K@named} | ‘Error(error_code){K@raw} ]; tracked status<S> bind(tracked(S) sock, sockaddr) [-S@raw]; Now users of the code lose access to the socket after calling bind. In order to regain that access, they must check the error code by deconstructing it. In either case, they regain the key, but in different states. If there is no error, execution can continue as normal. If there is an error, however, the code path must do something to put the key into the named state, otherwise there will be a type error at the join point, as demonstrated before.

17. Tying other variables • What about exposing internal pointers of tracked objects? • Examples: region memory allocation, returning an internal pointer of a struct. • Those values should be usable while the parent object is valid, but once the parent object is freed, these constituent objects also become invalid. • Solution: tie them to the same key as the parent

18. Region example • Region memory allocation puts objects into regions, and then the entire region is freed, instead of each individual object • References to objects in the region are no longer valid after the region is freed • Introduce new construct for region memory allocation: new(rgn) T [init] • Rgn is a region tracked by key R, and the whole expression returns a value of type R:T

19. Region example void okay() { tracked(R) region rgn = Region.create(); R:point pt = new(rgn) point {x=1; y=2;}; pt.x++; Region.delete(rgn); } void dangling() { tracked(R) region rgn = Region.create(); R:point pt = new(rgn) point {x=1; y=2;}; Region.delete(rgn); pt.x++; // error: key R not in held-key set } void leaky() { tracked(R) region rgn = Region.create(); R:point pt = new(rgn) point {x=1; y=2;}; pt.x++; // error: extra key R in held-key set }

20. Limitations • Would like to be able to put tracked objects inside other objects, and keep all the benefits • Currently, the container object’s type needs to be parameterized for every object it contains, so we can’t use anything growable • Parameterization means any value of the container type must also be tracked Tracked(A) Z a = …; Tracked(B) Y b = …; Tracked(A,B) ZYPair p= new Pair<A, B>(a, b)