FlowTalk

1. FlowTalk Harriet Truscott, Antranig Basman, CARET, University of Cambridge

2. FlowTalk – Use Cases Why did we build it? Many varied use cases for extremely flexible discussions software: • Student exercises with multiple rounds of feedback, different messages visible at different stages. Learning designer would like highest flexibility in designing workflow of teaching exercise. • Discussions for “outreach” – schoolchildren where highly flexible moderation policies necessary, including “pre-moderation”. Should also be possible via email. • Discussions amongst distributed research groups, self-organising into small “special interest groups”. Should be capable of functioning as a pure email list as well as web board.

3. FlowTalk - Technical goals Integration, Integration, Integration. • Learning environments move fast – should be capable of integrating as a “first-class” tool with everything out there now, or likely to be in the future. (Right now with Sakai, Stanford’s Coursework – future with LAMS, CopperCore, ??) Issues: Skinning User Idiom Installation AUTH

4. Skinning • Dealt with by RSF • In fact, historically, the reason RSF was created • Today, pure HTML templates are swappable dynamically • May even support direct upload of user’s own templates into the FlowTalk hierarchy • RSF presented in other talks, see wiki at http://www2.caret.cam.ac.uk/rsfwiki

5. Separate or In-JVM? • Pros: • Centralised deployment of ONE FlowTalk per site, with resulting smaller admin burden. • Increased stability – during host crash/restart, services remain available (email list &c) • Sakai footprint is already big enough! Tools in their own space is a Good Thing. • Cons: • Development is harder – more emphasis on serialisation/marshalling • Difficult access to host services (probably a good thing for an integration platform) • Performance – TCP and dispatch overheads per cycle.

6. Separate JVM We decided to go with separate JVM model. Of course, very much easier to rebundle a remote tool as a local one if required, reverse is much harder! Important to economise effort – having created ONE tool capable of remote interoperation with multiple VLEs/portals, try to reuse this effort by packaging as libraries: CaretAuth Abstracts user + groups model, together with hierarchical permissions structure and resolution. FlowTalk ClientPackage Operates user synchronisation protocol, request dispatch and consumer rendering request requirements.

7. FlowTalk architecture

8. Sakai Request dispatch cycle ToolSinkTunnelServlet DirectWebServiceFilter RequestDispatch FlowTalk HTML/HTTP request (Serialized ConsumerRequestInfo) RequestDispatch SiteService/ RealmService ) ( RemoteSyncServerUserManager InformationServlet * XML/HTTP request SakaiUIP ViewRender (copy bytes) * = At most once per request, and preferably never!

9. Notes on Dispatch Cycle • Must avoid “chatty” protocol, at most one “backchat” per cycle, preferably none. • The “general request” is NOT slowed down by backchat. Consumer info sent with request includes staleness timestamps, and client user structure is cached on FlowTalk side. • More problematic answering incoming email request on FlowTalk side – must have more slack “staleness” guarantees (e.g. 10 minutes lag) • Ideal case: ALL page rendering is done at producer, Sakai tool can start copying bytes to client with no latency. In fact achieve this with IKAT (see later)

10. Integration – Users/Resources • “CaretAuth” library abstracts permissions structure. Has successfully mirrored both CourseWork and Sakai structures, so will probably generalise again! • Permissions resolution is done on FlowTalk side with “equivalent effect” to Sakai resolution, no communications required. • FlowTalk maintains “global” resource tree, with one node per consumer. User information is scoped to nested resource nodes. • FlowTalk Permissions System

11. FlowTalk Global Resource Hierarchy Global resource root (per FlowTalk installation) Sakai Client 1 root Sakai Client 2 root Coursework Client 1 root Directly served resources Per-consumer info (URLs, IDs) CourseWork “Nexus” Sakai Sites Role/Realm info (as Groups, PermissionLists) CourseWork Section Forums/threads Sakai Pages/Placements (=“Forums”) Consumer per-resource Permissions (none for Sakai) Forums/threads

12. FlowTalk Permissions System • Is probably on the verge of become sentient and taking over the world • Permissions are resolved on • User/Group • Permission name (lock) • Resource type (forum/message) • Flow state (which have hierarchical relation!) • Permission resolution proceeds recursively up the resource hierarchy • Different user/group status at each level of the hierarchy! • Firstly because UserManager is scoped to the client • Secondly because indirect groupings (e.g. Sakai Site-wide Role) is mapped as a FlowTalk scoped group membership • Incredible flexibility, but needs to be presented very carefully to avoid complete bafflement (not least of us)

13. Notes on Resource Hierarchy/Events • FlowTalk stores resources of arbitary (MIME) types – “resource” is (generally) uninterpreted bytestream. • General resource metadata includes resource “state” forming basis of event-driven architecture leading to workflow. • Each change in event state fires off a (currently synchronous) event which propagates up the resource hierarchy being handled by listeners. • Event handlers may induce further changes in resource state, firing recursive notification. • An “event handler” is actually just a Spring bean, referred to by name.

14. Example Listeners (Workflows) • Default Workflow (simply moves message from “init” to “active” status) • Mail Notification (for all users with EMAIL_NOTIFY “permission”, sends out email for a message entering “active” status • Moderated Workflow (instead of moving from “init” to “active”, moves to “moderating” – which is a kind of “inactive” status) • PostMakesVisible Workflow (for any user making a post, adds them to the group “flowtalk:readers” within the parent forum)

15. FlowTalk Load Testing Results • Testing uncovered a variety of issues with Session leaks, and memory leaks in underlying libraries (all fixed now!) • As a result of its reliance on RSF, FlowTalk renders even complex or bulk views very quickly • Despite (or perhaps because of) using the filesystem, FlowTalk shows extremely high concurrency and low latency.