310 likes | 320 Views
This article discusses an approach for automatically maintaining relationships in flexible user interfaces (UIs) in media spaces, using a technique called "mumbling" to preserve the state and time relationships of different views. The author also introduces a tool called Ultraman for designing and implementing these UIs.
E N D
Constraints On TreesAnd A Forest Of Other Problems Ian SmithGeorgia Tech May 21, 1998
Talk Terrain Map Introduction Approach Flexible UIs Media Spaces “Mumbling” Abstraction State Preservation Time
Relationships JOB INTERVIEW Text Width Inner Width Inner X
High-Order Bit E A F C B B’ ... D D D ... D D C’
This is not a constraint. • This is automatic relationship maintenance. • Not numerical constraints. • Not logic programming. • Propagation-based systems • Not solved “from scratch”
Related Work • Sutherland’s Sketchpad • Borning’s DeltaBlue, SkyBlue, Indigo • Vander Zanden’s QuickPlan + studies • Hudson’s Eval/Vite • Myers and Guise’s KR solver • Patterson & Hill : AVL, etc. • Smalltalk’s active values
UI Applications • User interfaces are trees • Roughly: one tree is one “window”
A Bob Smith Steve McMan Ian Smith Alice Jones Tom Katt Kathy Wahl Chris Hopkins CTO Accounting President, CEO Distinguished Eng. CFO Director Of Eng. Director Of Finance Scenario: Org Hacker Org Hacker A& B Software Org Chart
Chris Hopkins Steve McMan Kathy Wahl Alice Jones Bob Smith Tom Katt Ian Smith Distinguished Eng. Director Of Eng. CFO President, CEO CTO Accounting Director Of Finance Bad Design A& B Software Org Chart Pilot Display Area
Improved Design Org Chart Reports To CEO, President (A. Jones) CTO (R. Smith) CFO (C. Tompkins)
The Set-Up • Two (or more) different “views” • Each view is created/maintained by a tree • Views are different, but related • Interactors are not one to one • Overall shapes are different • Views change over time… • Declare constraints!
Approach View 1 View2 Org Chart Reports To Ultraman President (John) VP of Eng. (Jane) Staff Eng. (Bill) VP of Finance (Fred) Window Pilot App Principal Sci. (Mary) Scroller Palette Scroller Tree Control ... Tree Node Line Rect Rect Tree Node Tree Node
Ultraman Design-Time • Tool for expressing relationships • Designer uses a pattern language • Language converted to a grammar • Grammar (and support code) compiled and linked with your app
Ultraman Run-Time • After each event… • Source tree is flattened • Flattened form is “parsed” • New result tree “generated” by matching patterns • Screen updated
State-Preservation • “Generating” interface B from A implies no state local to A • Each time B changes A’s state gets “trampled” • E.g. State of the “open-close” objects on Pilot interface
Incrementality • Given a small change in the source tree, make the correct, small change to the target tree C A A B C D E C
Approach • “Generate” a new target tree each time… • Identify the nodes that are the same • Preserve those nodes that haven’t changed C C C A A A A + = B C D E E E C C C
Sameness? • What does sameness mean? • Same location in the target tree? • Same type? • Result from the same constraint? • Result from the same object in the source tree? • Because they say so?
Solution • Precompute where sensible • Three conditions on sameness: • Produced from the same constraint • Produced from the same portion of the source • Compatible types • If two nodes are the same, they are merged to preserve state
Value Numbering • Bottom-up change metric run on the parse tree • Leave open possibility of VN on result tree • Technique derived from common subexpression elimination
Fun Things To Do • Good library of constraints • Grammar analysis • Toolkit for audio display • User-controlled error handling • Lazy toolkits
Flexible UIs • SubArctic -- Java UI toolkit • UI Toolkits are not a collection of widgets! • Building new interactions should be easy! • Built-in support for lenses (UIST ‘97) • Downloadable, dynamic styles (UIST ‘97) • Animation as a first class entity (UIST ‘93)
SubArctic and Me • ~60K lines of code • Responsible for design and impl. of major API sections • Bug fixing • Documentation • Release Engineering • Student management
Media Spaces • Media spaces : audio/video/textual environments for awareness • Privacy tradeoff • Cryptography helps... • Disturbance tradeoff
Mumbling • “Mumbling” is a privacy tradeoff • Remove large amounts of information content • Leave some interesting, but not sensitive, bits in the stream • Audio: 1 integer per 500ms with an amplitude approximation
Thanks • Special thanks to my hosts:Mike SalisburySteve Cousins • Thanks to Alex Zhao & Keith for giving me some screen shots. • Thank you for coming! • Thanks to Ultraman...