Defense Of Research

1. Defense Of Research Ian Smith

2. Overview • Motivational scenario • The problem • Approach • Tool explanation (example) • Local state problem • Contribution Summary • Related work

3. Scenario • Alice & Bob want to change the org chart together • They worked together before but now Alice travels more often... • Alice is remote, Bob is at his desk • Alice has a pilot, Bob has a workstation.

4. Workstation Org Chart App App Controls

5. Org Chart App On A Pilot?(Bad Idea 1) Pilot Display? App Controls

6. Org Chart App On A Pilot?(Bad Idea 2) Pilot Display

7. Pilot Org Chart Application Org Chart Reports To President (John) VP of Eng. (Jane) Staff Eng. (Bill) VP of Finance (Fred) Principal Sci. (Mary)

8. The Problem? • People want to work together, but have different devices • Two structurally different views of same data • UIs have different underlying structures • Views must be synchronized

9. Graphical Formulation Workstation Pilot Org Chart Reports To President (John) App Controls VP of Eng. (Jane) Staff Eng. (Bill) VP of Finance (Fred) Window Pilot App Principal Sci. (Mary) Scroller Palette Scroller Tree Control ... Tree Node Line Rect Rect Tree Node Tree Node

10. Workstation Pilot Org Chart Reports To Ultraman President (John) App Controls VP of Eng. (Jane) Staff Eng. (Bill) VP of Finance (Fred) Window Pilot App Principal Sci. (Mary) Scroller Palette Scroller Tree Control ... Tree Node Line Rect Rect Tree Node Tree Node My World

11. Problem Generalization • Given a UI tree A, compute a new UI tree B from A • If A is computed from a model, compute B from the same model Ultraman A B Model Ultraman Ultraman A B

12. Approach • End-programmer declares constraints on the structure of an interface • “Constraint” here is a function of a tree which computes a tree (not values!) • Constraints apply over an entire input tree • “When you see a node of type foo, create a node of type bar and attach to this structure…” • Constraints are applied dynamically, as the application runs

13. Using Ultraman • “Ultraman” is a tool and a runtime • Tool allows you to declare patterns and transformations • Pattern is a structural relationship of nodes in the source tree • Transformation is a computation to run when a pattern is found

17. Ultraman Tool Summary • Several ways to specify computations and structural patterns • Tool generates a lexer/parser specification which implements the pattern matching • Lexer/parser pair is built on ANTLR 2.20 • Generated parser calls into user code to implement transformations • Transformations are run after each event

18. Example App

20. Transformations? But…. Column Pull-Down Menu • Pull-Down menus have state: the currently displayed item. • If we regenerate the menu how do we get the right item displayed in the Pull-Down? Ultraman ... RadioButton* MenuItem MenuItem

21. Local State Problem • Tree T1 produces tree T2 by some transformation F. • User manipulates T2 in some way, modifying T2’s state. • We make a change to T1 and rerun transformation F producing tree T3. • How do we reconcile the freshly created T3 with the old, stateful tree T2?

22. Value Numbering • Nodes created by the transformations are given a “label” called a value number • The value number (VN) is a representation of “where and when” in the transformation node was created. • Nodes with the same VN are from the same “spot” in the transformation process.

23. Shadow Objects • Shadow objects are objects which “represent” interior nodes in the UI tree. • Shadow objects mimic the API of an interactor, but actually buffer calls • End programmers build dynamic structures underneath shadow objects. • Effectively, shadow objects build a “shadow” version of the interactor hierarchy during the transformation.

24. Grafting Algorithm • After the transformation is complete... • Walk the interface on the screen’s tree (T2) collecting pairs (VN,node). • Walk the shadow interface’s tree (T3) and compare VNs of its nodes to the table. • If a VN is in the table, exchange the corresponding nodes. • Replace the interface on the screen with the shadow.

25. Currently On Screen Shadow Version Result N1 (A) N1 N4 (A) N2 (B) N2 N5 (B) N6 (D) N3 (C) N6 Nodes To VN Table N1 A N2 B N3 C Grafting Algorithm Example

26. Issues With Grafting • Value numbering is a sensitive and bottom-up change metric. • Changes at the “top” of a tree can be missed due to shadow objects. • End-programmer can be surprised when objects which are created never appear! • Garbage collector can be a problem...

27. Contribution Summary • Developed a taxonomy for multi-viewed transformation (4 categories) & identified salient characteristics of multi-viewed transformation system (7 properties) • Implemented a specification tool for composing patterns and transformations

28. Contributions (cont’d) • Developed and implemented algorithms for converting specifications into grammars • Developed and implemented algorithms for handling the local state problem • Implemented 2 demo applications • Cards: Transform to a very different device • Contact: Transform someone else’s data model

29. Related Work • UI Software (excluding toolkits) • Patterson & Hill, Foley/Szekely, Olsen • Constraints • Borning, Hudson, Myers, Vander Zanden • Compilers/Parsing • Early, Parr, Graham & Glanville, Brotsky/Wills, Alpern • Transformation Systems • TXL, REFINE

30. Characteristics Of A Multi-Viewed Transformation System • Semantic Predicates • Aggregation • Multiple & recursive patterns • Input & output trees have different shapes • Control the progress of the pattern match • Multi-tree input • Take input from a model

31. Taxonomy Of Transformations • Filtering - projection (N to 1) • Aggregating - Computing a function of the entire input • Reordering - order change only (1 to 1) • Decorating - adding nodes to the input (1 to N)