Interaction Models I

1. Interaction Models I Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development March 9, 1999

2. Last Time: Heuristic Evaluation • A “discount” usability testing method • UI experts inspect an interface • prototype initially • later, the full system • Check the design against a list of design guidelines or heuristics • Present a list of problems to the UI designers and developers, ranked by severity • See homework assignment (Due Mar 18)

3. Outline • Windowing systems • Events overview • Event dispatching and handling • For more information, see Olsen 98, Developing User Interfaces

4. Windowing Systems • Provide infrastructure to support common UI-related services • Provide functionality for • Input/Output device handling • Which window gets which input/output • Window manager • create and organize windows • implement interaction in those windows

5. Windows • Top level windows known as root windows • provide an abstraction to separate applications • windowing system arbitrates resources among these • Each root window belongs to an app. • all descendant windows belong to same application • not used by OLE (ActiveX) Adapted from slide by James Landay

6. Networked Windowing Systems • XWindows designed to allow applications to run on remote machines • Uses client-server model • (X reverses standand usage of client/server terminology) Network X Server std system software Client app software User Adapted from slide by James Landay

7. XWindows • Note backwards terminology • User is on “server” not “client” • X Server • interprets X commands and can send events • determines which window receives events and forwards over network to proper client • X Client • software interface to X (Xlib) • assembles the output from Xlib routines into packets for transmission to server Adapted from slide by James Landay

8. Events • An event is something “interesting” that happens in the system • Mouse button goes down • Item is being dragged • Keyboard button was pressed Adapted from slide by James Landay

9. Window Events • User interacts with input device • action translated into software events • must be distributed to the appropriate window • Events contain information on • type • mouse position or character key • the window the event is directed to Adapted from slide by James Landay

10. Input Events • Mouse button events • depress/release mouse key • modifier (shift keys, etc.) • double click (X doesn’t have this -> fakes it) • Mouse movement events • implement painting with mouse • mouse drag • Mouse enter and exit events • e.g. if you entered / exited a button region Adapted from slide by James Landay

11. Button Events Button mouse enter mouse exit Button (But using mouse move events within a button is unnecessary, due to the semantics of buttons.) Adapted from slide by James Landay

12. Events (cont.) • Keyboard events • translate raw “scan codes” into ASCII Adapted from slide by James Landay

13. Events (cont.) • Windowing events on window • creation / destruction • opening / closing • iconifying / deiconifying • selection / deselection • resize • redraw • (window manager tells the application to redraw within a certain region; the application does the actual redrawing) Adapted from slide by James Landay

14. Widgets that can register specific kinds of events close box title bar folder scroll bar size control Adapted from slide by James Landay

15. Major Issues • How to decompose the UI into interactive objects (widgets)? • How to distribute inputs to the interactive objects? • Contrast • Sequential (standard) program flow • Event-driven program flow Adapted from slide by James Landay

16. Sequential Programs • Program takes control, prompts for input • Examples include • command-line prompts (DOS, UNIX) • LISP interpreter • The user waits on the program • Program tells user it’s ready for more input • User enters more input

17. Sequential Programs (cont.) • The user waits on the program • 1. Program tells user it’s ready for more input • 2. User enters more input • 3. Program responds and goes back to 1. • But how to model the many actions a user can take? • For example, a word processor? • Need to do editing, inserting, etc. Adapted from slide by James Landay

18. Event-Driven Programming • Instead of the user waiting on program, have the program wait on the user • All communication from user to computer is done via “events” • An event is something “interesting” that happens in the system • Mouse button goes down • Item is being dragged • Keyboard button was pressed Adapted from slide by James Landay

19. Interactor Tree • Decompose interactive objects into a tree • based on screen geometry of objects • use nested rectangles • Used for dispatching events • Events are dispatched (sent) based on code associated with the widget • The code responds appropriately to the event • Variety of methods for dispatching events • Return to this later Adapted from slide by James Landay

20. Interactor Tree Display Screen “F:\cs160\Public” window Inner Window title bar horizontal scroll bar contents area “CDJukebox” folder “Home Ent…” folder … size control … “Web Newspaper” window … Adapted from slide by James Landay

21. 6 9 - 3 7 4 0 1 2 + 5 8 Interactor Tree Display Screen 93.54 Outer Win [white] ????? = ENT Adapted from slide by James Landay

22. 6 9 - 3 1 0 4 7 + 5 2 8 Interactor Tree Display Screen 93.54 Outer Win [white] Inner Win [green] Result Win [tan] Result String Keypad [grey] = button = ENT - button + button 0 button Adapted from slide by James Landay

23. Main Event Loop • Main event loop Initialization While (not time to quit) { Get next event E Dispatch event E } • The meat of the program is in the code that handles the “dispatch” Adapted from slide by James Landay

24. Event Queues • Input events are placed in a queue • Ensures events are processed in order • Main event loop removes them from the queue (get-next-event) & dispatches for processing Mouse move (22, 33) Mouse move (40, 30) Mouse down left (45, 34) Mouse up left (46, 35) Adapted from slide by James Landay

25. Event Queues (cont.) • Can use ignore unwanted events • e.g., ignore mouse moves in a forms-based program • just get enter/exit events • however, do want to track mouse moves in a drawing program Adapted from slide by James Landay

26. 9 6 - 3 0 7 4 1 2 + 5 8 Event Dispatch Dispatch (event E) { switch (E.window) { ... case FIVE-KEY: if (E.type == left-down){ cur = 5 + 10*cur; display (cur); last-op = NUMBER; } ... 0 Hit the ‘5’ key = ENT Adapted from slide by James Landay

27. 9 6 - 3 0 7 4 1 2 + 5 8 Event Dispatch Dispatch (event E) { switch (E.window) { ... case TWO-KEY: if (E.type == left-down) { cur = 2 + 10*cur; display (cur); last-op = NUMBER; } ... 5 Hit the ‘2’ key = ENT Adapted from slide by James Landay

28. 6 3 - 9 1 4 0 7 8 2 + 5 Event Dispatch Dispatch (event E) { switch (E.window) { ... case ENTER-KEY: if (E.type == left-down){ push (cur); cur = 0; last-op = COM; } ... 52 = ENT Hit the ‘enter’ key Adapted from slide by James Landay

29. 9 6 - 3 0 7 4 1 2 + 5 8 Event Dispatch Dispatch (event E) { switch (E.window) { ... case SIX-KEY: if (E.type == left-down) { cur = 6 + 10*cur; display (cur); last-op = NUMBER; } ... 0 Hit the ‘6’ key = ENT Adapted from slide by James Landay

30. 6 3 - 9 1 4 0 7 8 2 + 5 Event Dispatch Dispatch (event E) { switch (E.window) { ... case PLUS-WIN: if (E.type == left-down){ if (last-op == NUMBER) push (cur); result = pop() + pop(); push (result); display (result); cur = 0; last-op = COM; } 6 = ENT Hit the ‘+’ key Adapted from slide by James Landay

31. Dispatching Events • If user scrolls the text, the software must: • direct the mouse events to the scroll bar • update the scroll bar display during the drag • notify the text editing window it needs to scroll itself so that the text appears to have moved Adapted from slide by James Landay

32. Dispatching Events (cont.) • Algorithm selects the bottom-most, front-most region in the interactor tree • Called bottom-first event dispatching • scroll bar or contents over outerwin (bottom-most) • scroll bar over contents (front-most) • Advantage: • each window need only consider its own events • simple • disadvantage: • difficult to impose a high level of control • inflexible Adapted from slide by James Landay

33. Dispatching Events (cont.) • Top-down event dispatching • events passed to top-most, front-most window • it dispatches to one or more of its children Adapted from slide by James Landay

34. Event Focus • Where should keyboard events go? • mouse-based • attach mouse position to all key events and dispatch events in the same way as mouse events • click-to-type • send all keyboard events to last window where mouse-down occurred Adapted from slide by James Landay

35. Event Focus • Mouse focus • scrollbar example: retain the focus on the scrollbar widget until the mouse button is released • compensates for difficulty of keeping the mouse within the narrow scrollbar Adapted from slide by James Landay

36. Event Handling • Event tables (in the early days…) • indexed by event types (integer from 0 - 255) • hold pointers to functions that handle each event • one table per / window • lots of things to maintain when attached to a widget that you want to make reusable • Callbacks • each kind of widget defines a set of named callbacks which will be run when the widget receives an appropriate event Adapted from slide by James Landay

37. Callback Example • How do we notify text window to scroll when the scroll bar is moved? • create a vertical scroll bar widget • write a callback procedure which has code to notify text windows of their new position • register the callback as being the program to invoke when the scroll bar is moved • register the text window as the data for the callback • so the system knows which window to scroll Adapted from slide by James Landay

38. Summary • Windowing systems • software to manage and arrange windows • keeps track of current focus • software to support handling of user-created events • complex GUIS are built up of hierarchically nested windows • Events • associated with various types of input devices and actions • are handled one by one from a queue