Very Quick Introduction to Organization of UI Software

1. Very Quick Introduction to Organization of UI Software The basics of what goes into a user interface (mostly GUIs) Tasks and components to implement them

2. The User Interface • Typically want to think of “UI” as only one component of an overall system • The part that “deals with the user” • Distinct from the “functional core” (AKA the “application”)

3. Separation of UI from “Appl” • Really good reasons to want separation of UI from “application” • In general want “separation of concerns” • Modularity (good software design) • Different expertise needed • Don’t want to iterate the whole thing

4. Unfortunately this is typically very hard to do in practice • More and more of interactive programs are tightly coupled to UI • In some cases everything is “in” the UI • Why? • Need to structure around user concepts • UI structure “sneaks into” application • Tight coupling can offer benefits to user (better feedback)

5. Separation of concerns is a central theme of UI org • A continual challenge • A continual tension and tradeoff • Real separation of UI from application is almost a lost cause

6. Appl UI UI tasks • So far have: • Clearly there is more structure

7. Input Appl Inter Appl UI Core Output UI tasks • Basic parts of UI

8. Input Appl Inter Appl UI Core Output UI tasks • Basic flow

9. Canonical code structure for an interactive program • Find appropriate object to deliver the input to • Object decides what to do with it based on: • What it is • What state it is in Initialize(); Repeat Evt := Wait_For_Input(); Dispatch_Input(Evt); If something_has_changed Then Redraw_All(); Until time_to_exit; Ask each object whose appareance might have changed to redraw itself (based on what it is and what state it is in)

10. This is the basic “Event/Redraw Loop” Initialize(); Repeat Evt := Wait_For_Input(); Dispatch_Input(Evt); If something_has_changed Then Redraw_All(); Until time_to_exit; Used in some form by almost all interactive systems

11. “Most of the Work” Input Appl Inter Appl UI Core Hardware OS etc. Output Layered Drawing/Windows Input Abstraction I/O Devices Need to use system infrastructure to implement • Layered system components (for GUI) Toolkit Window Sys OS

12. Toolkit Input Appl Inter Appl Window Sys UI Core Hardware OS OS etc. Output Need to use system infrastructure to implement • Unfortunately market forces give us: OS

13. Input Appl Inter Appl UI Core Hardware OS etc. Output Need to use system infrastructure to implement • But conceptually these are the right layers Toolkit Window Sys OS

14. Input Appl Inter Appl UI Core Output Look mostly at toolkit level • Tasks remain:

15. How do we connect these disparate parts into a working whole • Tempting to architect systems around these boxes • A module for input, one for output, one for application interface • Things like this have been tried • “Seeheim model” • Didn’t work real well

16. Architectures with “3 big boxes” don’t work well because... • Modern (“direct manipulation”) interfaces tend to be collections of quasi-independent agents • Each “object of interest” is separate • e.g. a button • produces “button-like” output • acts on input in a “button-like” way • etc. • Each object does its tasks based on • What it is • What its current “state” is • Context from prior interaction or application

17. Leads to object-based architecture • Interactor objects • AKA components, controls, widgets • Each object implements each aspect • In a way that reflects what it is • Objects organized hierarchically • Normally reflecting spatial containment relationships • “Interactor trees”

18. Toolkit Infrastructure Interactor Tree Operation Interface represented by tree Application

19. Toolkit Infrastructure Interactor Tree Operation Input Application

20. Toolkit Infrastructure Interactor Tree Operation Application response (manipulate tree) Application

21. Toolkit Infrastructure Interactor Tree Operation Objects update selves & declare damage Application

22. Toolkit Infrastructure Interactor Tree Operation Redraw (top-down traversal) Application

23. Toolkit Infrastructure Interactor Tree Operation Complete “input-redraw-cycle” again Application

24. Challenge: separation of concerns • Challenge is doing all this different stuff in a single object without creating a hopelessly large and complicated beast • Three general approaches • several models in each • not mutually exclusive (can / should use multiple) • Composition • Inheritance • Aggregation

25. Composition • Put together interactive objects at larger scale than interactors • Container objects • e.g., row and column layout objects • Containers can also add input & output behavior to things they contain

26. Composition (containers) • Can also have more sophisticated containers that change input/output • Composition approach separates concerns into different interactors

27. Approaches at the interactor level • Inheritance • all concerns in one object • inherit / override them separately • works best with multiple inheritance • example: draggable_icon • inherit appearance from “icon” • output aspects only • inherit behavior from “draggable” • input aspects only

28. Inheritance • Don’t have multiple inheritance in most languages (i.e. java) • but can still (partially) take this approach • Inheritance tends to give tighter coupling between aspects • together in the code, no boundaries • Inheritance is the most common approach

29. View Model Cntr Another object level approach:Aggregation • Actually separate out different concerns into separate objects • Treat collection as “the interactor” • Classic architecture: “model-view-controller” (MVC) • from Smalltalk 80

30. View Model Output Controller Input Model-View-Controller • Each is separate object

31. Model-View-Controller • Model • the “underlying” or “application” information we interact with • MVC takes approach of “editing” this information • Fits with direct manipulation interface paradigm (model is object user manipulates, but not representation)

32. Model-View-Controller • Model • Simple examples • text editor: model is text string • slider: model is an integer • Model is “data only” • no input or output aspects • but may include behavior (semantics)

33. Model-View-Controller • View • mechanism needed to map model data to rendition (view / display) • all output aspects here • when model changes, view object is informed • view arranges to update screen • Declare damage • Redraw when requested

34. Model-View-Controller • Controller • listens to user input • translates into changes to model • all input aspects here • Controller almost always has to “talk to” view • Why?

35. Model-View-Controller • Controller almost always has to “talk to” view • need geometry of output to interpret input (e.g., picking) • need to do feedback! • As a result, VC tend to be very tightly coupled

36. Model-View-Controller • In theory should be able to plug different views and controllers • good property • in practice VC tend to be written together and be too tightly coupled • Typical modern view of MVC • combine VC into one object • M(VC)

37. Tasks again in more detail • Core functions (cross cutting) • Hierarchy management • Again: will be trees of objects • Geometry management • coordinate systems • bounds • Object status / information management • Visible, enabled, selected, …

38. Tasks again in more detail • Output • Damage management • knowing what needs to be redrawn • Layout • establishing size and position of each object • (Re)drawing

39. Tasks again in more detail • Input • Picking • Figuring out what objects are “under” a screen point • Event dispatch, translation, handling • A lot of the work is in here • In a typical toolkit “handling” is often majority of the code you write

40. Tasks again in more detail • Application interface • (Not very well developed) • Callback model • Command objects