230 likes | 360 Views
Past, Present and Future of User Interface Software Tools. Brad A. Myers (work with Scott E. Hudson and Randy Pausch) Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University http://www.cs.cmu.edu/~bam bam@cs.cmu.edu. Introduction.
E N D
Past, Present and Future ofUser Interface Software Tools Brad A. Myers (work with Scott E. Hudsonand Randy Pausch) Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University http://www.cs.cmu.edu/~bam bam@cs.cmu.edu
Introduction • User Interface Software Tools • Help developers design and implement user interfaces • Focus on Tools, but influenced by future UIs • Today’s tools are highly successful • Window Managers, Toolkits, Interface Builders ubiquitous • Most software built using them • Arebased on HCI researchBrad A. Myers. “A Brief History of Human Computer Interaction Technology.” ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54. http://www.cs.cmu.edu/~amulet/papers/uihistory.tr.html • Future tools must be different
What Worked • Window Managers and Toolkits • Event Languages • Graphical, Interactive Tools • Component Architectures • Hypertext • Object Oriented Programming
Window Managers • Multiple (tiled) windows in research systems of 1960’s: NLS, etc. • Overlapping introduced in Alan Kay’s thesis (1969) • Smalltalk, 1974 at Xerox PARC • Successful because multiple windows help users manage scarce resources: • Screen space and input devices • Attention of users • Affordances for reminding and finding other work
Toolkits • A collection of widgets (“controls”) • Menus, scroll bars, text entry fields, buttons, etc. • Toolkits help with programming • Help maintain consistency among UIs • Key insight of Macintosh toolbox • Successful partially because address common, low-level features for all UIs
Event Languages • Create programs by writing event handlers • Many UIMSs used this style • Univ. of Alberta (1985), Sassafras (1986), etc. • Now used by HyperCard, Visual Basic, Lingo, etc. • Toolkits with call-backs or action methods are related • Advantages: • Natural for GUIs since generate discrete events • Flow of control in user’s hands rather than programmer’s • Discourages moded UIs • Won’t work well in future
Graphical Interactive Tools • Create parts of user interface by laying out widgets with a mouse • Examples: Menulay (1983), Trillium (1986), Jean-Marie Hullot from INRIA to NeXT • Now: HyperCard, Interface Builders, Visual Basic’s layout editor, resource editors, “constructors” • Advantages: • Graphical parts done in an appropriate, graphical way • Accessible to non-programmers
Component Architectures • Create applications out of components which are separately developed and compiled • In UI software, each component controls an area of the screen • Example: drawing component handles picture inside a document • Invented by Andrew research project at CMU (1988) • Now: OLE, OpenDoc, ActiveX, Java Beans
Hypertext • Ted Nelson named it in 1965 and developed a Hypertext system at Brown University • Important systems: NLS (1967), Hyperties (1986) • World-Wide Web • Phenomenal success due to: • Ease of use of Mosaic browser • Support for embedded graphics • Support for easy authoring
Object Oriented Programming • Success of OO owes much to UI software field • Popularized by Smalltalk • GUI elements (widgets) seem like objects • Have state, accept events (messages) • Rise parallels GUIs • C++ with Windows 3.1
What Hasn’t Caught On • User Interface Management Systems • Separation of UI and application is harder than expected • Formal Language-Based Tools • Grammars and Transition networks are not appropriate for direct manipulation • Constraints • Declare once, system maintains • Can be unpredictable and hard to control • Model-Based and Automatic Techniques • Results are not as good as hand-created • Sometimes, no savings of effort
Future Prospects and Visions • Important Trends • Ubiquitous Computing • Move to recognition-based interfaces • End-user customization and scripting
Ubiquitous Computing • Computation embedded in many kinds of devices • Digital pagers and cell phones, Palm Pilots, CrossPads, laptops, wall-size displays, “smart” rooms • Next wave: easy communication with radio • E.g., BlueTooth: www.bluetooth.com • Significant Implications for tools • Moving target problem • Tools for coordinating multiple, distributed,communicating devices • “Multi-computer” user interfaces
More problems: Varying Input and Output • Today’s Desktop screens vary by a factor of 2.5 in size and a factor of 4 in pixels • Tomorrow’s screen will vary by factors of 100 in size and a factor of 625 in pixels • Cell phone to Stanford’s wall (3796 x 1436 pixels)
Varying I/O Capabilities, cont. • Interaction techniques for desktop will not work • No room on small devices • Can’t reach menubar on wall-size devices • Want to run same application on different devices
Need for Prototyping Devices • User interface will be in hardware • Rapid design and prototyping needed for hardware • Pragmatics and usability cannot be evaluated from a simulation on a screen
Move to recognition-based interfaces • Speech, gestures, camera-based vision • Multimodal interaction • User will pick which modality to use • Use multiple modalities at same time • Today, programming these requires knowing about Hidden-Markov Models, grammars, feature vectors, etc. • Need tools to hide these complexities
Fundamental Differences ofRecognition-based UIs • Input is uncertain • Recognition can make errors • Requires monitoring, feedback, correction • Do not produce discrete events • GUI event model no longer works • Interpreting input requires deep knowledge of data • Context of the application • “Move the red truck to here” • Need a architecture based on accessible application data structures • “Reflection”, “Open Data Model”
More Assumptions of Today’s Tools • Skill and Dexterity of users • Older users • Makes single, fixed library of widgets untenable • Non-overlapping and opaque components • Preclude translucency, magic lens interactions
Operating Systems Considerations • What is in the OS? • Window Manager? Toolkit? Communication? Scripting facilities? • Need ever increasing services for applications • Need more access to low-level information • E.g., hardware buttons, whether on network • Ideally, API to support competition and research into these components
Conclusions • Research in tools necessarily trails innovation in UI design • Due to consolidation on desktop metaphor, significant progress in tools • UI design poised for radical changes • New opportunities and challenges for tools
Talk Given Using • Slide Show Commander • To be released commercially by: • Available soon! • Developed as part of thePebbles Projecthttp://www.cs.cmu.edu/~pebbles Synergy Solutions, Inc. http://www.synsolutions.com/
Thank you! Past, Present and Future ofUser Interface Software Tools Brad A. Myers (work with Scott E. Hudsonand Randy Pausch) Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University http://www.cs.cmu.edu/~bam bam@cs.cmu.edu