Interaction

1. Interaction Jing Li CPSC 533C March 3, 2003

2. Overview • Toolglass and Magic Lenses: The See-Through Interface (1993) by Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, Tony D. DeRose • SDM: Selective Dynamic Manipulation of Visualizations (1995) by Mei C. Chuah, Steven F. Roth, Joe Mattis, John Kolojejchick

3. Toolglass and Magic Lenses • The see-through interface that can appear as though on a transparent sheet of glass between the application and the traditional cursor. • These tools create spatial modes that can replace temporal modes in user interface systems.

4. Introduction • Toolglass widgets: semi-transparent interactive tools that are used in an application work area, and appear on a virtual sheet of transparent glass called a Toolglass sheet. • The user can line up a widget, a cursor and an application object in a single two-handed gesture. • A widget that has been stretched across the entire work area creates a command mode. • User can move sheet over application object or move the object to a widget (trackball can control both sheet and scrolling). Figure 1. Click-through buttons. (a) Six wedge objects. (b) Clicking through a green fill-color button. (c) Clicking through a cyan outline-color button

5. Introduction (Cont.) • Many widgets can be placed on a single sheet • Widgets and lenses can be composed to create a number of specialized tools from a basic set(e.g. an outline color palette over a magnifying lens, to point to individual edges) Figure 2. A sheet of widgets. Clockwise from upper left: color palette, shape palette, clipboard, grid, delete button, and buttons that navigate to additional widgets Figure 3. An outline color palette over a magnifying lens

6. Related Work • Multi-Handed Interfaces • Movable Tools • Menus that pop up at the cursor position • Toolglass sheets can be moved without tying up the cursor • Transparent Tools • Make menus over the work area transparent • Viewing Filters • Magic lenses are visual filters that may be used to change type of views or reveal hidden information.

7. Examples • Shape and Property Palettes Figure 4. Shape palette. (a) Choosing a shape. (b) Placing the shape. Figure 5. Font face palette. The word "directly" is being selected and changed to bold face.

8. Examples (Cont.) • Clipboards Figure 6. Symmetry clipboard. (a) Picking up an object. (b) Rotated copies appear. (c) The copies are moved and pasted. Figure 7. Fill-color rubbings. (a) Lifting a color. (b) Moving the clipboard. (c) Applying the color.

9. Examples (Cont.) • Previewing Lenses Figure 8. An achromatic lens over a drop shadow lens over a knotwork. (Knotwork by Andrew Glassner)

10. Examples (Cont.) • Selection Tools Figure 9. Vertex selection widget. (a) Shapes. (b) The widget is placed. (c) A selected vertex. Figure 10. The local scaling lens that shrinks each object around its own centroid (Tiling by Doug Wyatt)

11. Examples (Cont.) • Grids Figure 11. Three grid tools

12. Examples (Cont.) • Visualization Figure 12. Gaussian curvature pseudo-color lens with overlaid tool to read the numeric value of the curvature. (Original images courtesy of Steve Mann)

13. Implementation of Toolglass Sheets • Multi-Device Input and Screen Refresh • Handles simultaneous input from two pointing devices and updates the screen after multiple simultaneous changes • Filtering Input Through Lenses and Widgets • Modifies pointing events as they pass through widgets • Filtering Output Through Lenses and Widgets • Modifies graphical output as it passes up through each widget

14. Implementation of Magic Lens Filters • Recursive Ambush • catch the low level drawing calls • typically a wrapper around the graphics context • and modify them, and then call the original graphics primitive • e.g. modify “DrawLine” to set the color to “red” • works transparently across all applications • but complicated to make a new lens and debug

15. Implementation of Magic Lens Filters (Cont.) • Model-In Model-Out • create new objects and transform them • can be saved and used by any lenses over it • very powerful, but • must allocate storage • cross application is an issue

16. Implementation of Magic Lens Filters (Cont.) • Reparameterize & clip • Modify global parameters to drawing • Redraw the model, but clipped to the boundary shape of the lens • Best example: scaling

17. Implementation of Magic Lens Filters (Cont.) • Multiple model types need to be supported for cross-application functionality. Achieved by: • Type conversion: convert the model to be of the type required • Type tolerance: unchanged if does not understand a model

18. Toolglass & Magic Lens Conclusions • See-through interface based on spatial modes • Structured well for two-handed interaction • May be integrated into any screen-based applications and tools

19. SDM: Selective Dynamic Manipulation of Visualizations Mei C. Chuah, Steven F. Roth, Joe Mattis, John Kolojejchick (Carnegie Mellon University)

20. Motivation • Need new interactive techniques for 2D and 3D visualizations. We want: • Selective: a high degree of user control • Dynamic: interactions all occur in real time, and interactive animation • Manipulation: Users can directly move and transform objects to perform different tasks • This set of techniques is called SDM

21. Current Static Visualization • Users cannot focus on specific object sets in detail while keeping them in the larger context • Clutter and occlusion (hidden data) in dense information space • Difficult to give a sense of scale: some objects may be dwarfed in the scale used for the entire data set (green objects in [Figure 1]) • No tools provided to classify data sets and save those classifications • Difficult to compare quantities when objects are not spatially contiguous (e.g. when objects are at different distances from the user)

22. SDM Sample • SDM deals with these issues • Selection and control methods rely on objects rather than spaces.

23. Sample: Supply Distribution Network • Cylinders (red) : supply centres • Rectangular bars (purple) : shelters • Lines (black) on the floor plane : major routes • Heights of cylinders and bars : quantities available at supply centres and needed by shelters • Selected objects : green

24. SDM Components • Object Centered Selection • Select objects by clicking or using sliders to place data constraints • Object sets may be named, saved and used later from a scrollable menu • Object sets need not be made up of homogeneous types (e.g. a set may contain supply centers, shelters, and routes)

25. SDM Components (Cont.) • Dynamic and Flexible Operations • Directly manipulate through object handles • Attach a handle to an object, and push or pull on it: causes the object to expand, shrink or move continuously • Objects change with motion of handles (A) controls the radius of cylinder object (B) controls object height(C) controls bar width • Arrow handles: shift

26. SDM Components (Cont.) • Object Constraints Context persistence • SDM maintains some relationship between the focus objects set and its environment. Set-wide operations • If you can move or scale one object in a focus set, you can move or scale any other object in this set.

27. SDM Feedback Mechanisms • Clearly Identify the Selected Set • The selected set is painted differently (green) • A white grid may be drawn beneath all selected objects to indicate positions and spread • Maintain Scene Context • Used to maintain context when objects are displaced • Object ‘body’ (green) and object ‘shell’ (white)

28. Feedback Mechanisms (Cont.) • Maintain Temporal Continuity • Use animation to help users to perceive changes to the scene • Maintain Relationships Between the Selected Set and the Environment • Keep a scale of the differences on screen, for example, the ratio axes technique (next slide) • Allow Objects to be Easily Returned to Their Home Positions

29. Ratio Axes Technique Scale of the environment (black) & the selected set (red)The left is 1:1; the right is 1:8

30. SDM Solutions • Focusing on a Select Set of Objects while Keeping Scene Context • Objects can be painted distinctly and the width can be increased • Users may shift all focus objects to the front of the scene

31. SDM Solutions (Cont.) • Viewing and Analyzing Occluded Objects • Selected object set is elevated to solve occlusion problem

32. SDM Solutions (Cont.) • Reduce the heights of unconcerned objects to zero • Make all unconcerned objects very thin

33. SDM Solutions (Cont.) • Viewing Different Sets of Elements Based on Different Scales • Ratio axes technique (mentioned above) • Interactively Augmenting the Visualization with New Classification Infomation • Add width and color, save as a set and apply them later (i.e. identify important shelters)

34. SDM Solutions (Cont.) • Comparing the Patterns, Widths, and Heights of Objects • Draw a line of reference in the scene plane and move any set of objects to the reference line • Compare data trends among multiple sets

35. SDM System Architecture

36. SDM Conclusions • Good interactive techniques deals with the issues from current static visualizations • Visualization is clear, precise and easy to understand • The goal of SDM is to provide users with enough tools and flexibility (multiple alternative solutions) that they can solve a wide spectrum of data analysis tasks.