330 likes | 531 Views
Scriboli: High Performance Pen Interfaces. Ken Hinckley Patrick Baudisch Gonzalo Ramos Francois Guimbretiere Microsoft Research. Scriboli Design Goals. Speed without keyboard hotkeys Cognitive footprint diminishes with use Recall / declarative procedural skill
E N D
Scriboli: High Performance Pen Interfaces Ken Hinckley Patrick Baudisch Gonzalo Ramos Francois Guimbretiere Microsoft Research
Scriboli Design Goals • Speed without keyboard hotkeys • Cognitive footprint diminishes with use • Recall / declarative procedural skill • Minimal demands on visual attention • Repeatable motions for the user • Predictable & dependable system behavior • Expressiveness • Support many command structures & apps • Economy of design • only a few things to learn
SKETCH – Brown University • Adding 1 more gesture breaks everything • hard to learn, gestures specific to 3D editing
Recognize Pen Gestures? • Classic problem: “ink” or “gesture” ? • Hard problem: any ‘gesture’ could be ink • Inferred decision comes too late • real-time interactive feedback difficult • Make it a physical skill • All we need to solve this is ONE BIT of info • physical button on bezel • habit-forming, deterministic, low attentional demand • nonpreferred hand, no trip to “lasso mode”
Phrase Structure for Scriboli Scope Command Parameters Separator (start) Separator Scope/Command Separator Command/Parameters Separator (Done)
Phrase Structure for Scriboli Scope Command Parameters Separator (start) Press “Gesture” Separator Scope/Command Separator Command/Parameters Separator (Done) Fast and reliable [Sellen 90, 92] Can be chunked by users
Phrase Structure for Scriboli Scope Command Parameters Separator (start) Press “Gesture” Separator Scope/Command Pigtail Separator Command/Parameters Separator (Done) Fast, based on muscle “memory” Reliable and easily chunked by users
Phrase Structure for Scriboli Scope Command Parameters Separator (start) Press “Gesture” Separator Scope/Command Pigtail Separator Command/Parameters Crossing boundary Separator (Done) • Merge command selection and direct manipulation • FlowMenu [Guimbretiere 00], Control menu [Pook 00]
Phrase Structure for Scriboli Scope Command Parameters Separator (start) Press “Gesture” Separator Scope/Command Pigtail Separator Command/Parameters Crossing boundary Separator (Done) One stroke for scope, command, and parameters Non-modal system (quasi-modal)
Delimiters for scope selection + marking • A delimiter is “something different” • lexical structure of stroke • e.g. self-crossing gesture stroke • DEMO / VIDEO :delimiter techniques
Results – Completion Time • Left bar – main block; right bar – repeated block • Dotted bars are Pigtail2 design iteration Button Handle Pigtail Pigtail2 Timeout
Results – Learning Effects • For the 24 trials of Repeated Invocation Block
Results – Error Rate • Error rate of selecting wrong marking direction Button Handle Pigtail Pigtail2 Timeout
Punct. space capitals period comma object verb indirect obj. Written text separate words sentence start sentence end delimits clause “The cat sat on the mat” Scriboli Equivalent Pen up/pen down Gesture button down Gesture button up Pigtail (delimiter) Scope (lasso, line, …) Marking menu Crossing manip. phase Summary: Grammar for pen input • Scriboli proposes fundamental building blocks • Links together object, verb, & indirect object in fast, fluid, unambiguous command phrases
Stitching: Pen Gestures that Span Multiple Displays Microsoft Research Ken Hinckley Gonzalo Ramos Francois Guimbretiere Patrick Baudisch Marc Smith
Wireless Network 2004 = Horseless Carriage 1904? • Are Wireless Networks really just wired networks without the wires? • Or are they something completely different?
Users need techniques to intuitively form bridges between devices How do users name the devices to connect? What is type / purpose of the connection? Parameters? (Who copies what, to where?) Name That Device
Pen stroke that spans displays Move the pen Cross over bezel Finish stroke on nearby tablet System infers connection Stitching
AVI’04 ACM Advanced Visual Interfaces 2004 VIDEO
Recognizes timing & dynamics of pen trajectory There is nothing special about the pen! Wireless signal strength determines nearby devices Tablet 2 Tablet 1 Δt Establishing a Connection
Fits a line to absolute coords of user’s pen stroke Automatic Screen Calibration
SociologyProxemics: How people use space Invisible bubble that surrounds an individual How Do Users Share Physical Space?
SociologyProxemics: How people use space Invisible bubble that surrounds an individual 6. How Do Users Share Physical Space? • Homework assignment: • Sit right next to someone at airport (when it is not necessary to do so) • Time how long it is before the other person leaves
SociologyProxemics: How people use space Invisible bubble that surrounds an individual 5. How Do Users Share Physical Space?
Don’t require contact : touching is taboo “Do they have to be right next to one another?”: 10-40cm “wide screen would be nice for collaboration” “no two faces trying to peek at only one screen” Proxemic Lessons for Stitching
Don’t require contact “Do they have to be right next to one another?”: 10-40cm “wide screen would be nice for collaboration” “no two faces trying to peek at only one screen” Gives users flexibility to be involved, or not: mood, task, … Intimate Spaces: Combining screens. For close collaboration with friend or trusted colleague Personal Spaces: Tablets can be separated by up to 30” yet still possible to stitch to give files to colleague, etc. Social Spaces: Once connected, “transporter” can be used to give files to a user beyond arm’s reach Proxemic Lessons for Stitching
Multiple Devices: Cooperative Stitching User 1 User 2 User 4 User 3 Ongoing work
Multiple Devices: Cooperative Stitching User 1 User 2 User 4 User 3 Ongoing work
Handle vs. Pigtail • Handle can get in the way • Adds some visual clutter • Must check to be sure landed on handle • Pigtail handles more than one scope elegantly • Self-referential gesture