1 / 29

User Interface Toolkits and End User Programming for UIs

User Interface Toolkits and End User Programming for UIs. Vishal Dwivedi Institute for Software Research Carnegie Mellon University vdwivedi@cs.cmu.edu Advanced User Interface Software.

willis
Download Presentation

User Interface Toolkits and End User Programming for UIs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. User Interface Toolkits and End User Programming for UIs Vishal Dwivedi Institute for Software ResearchCarnegie Mellon University vdwivedi@cs.cmu.edu Advanced User Interface Software

  2. “End-user programming enables end users to create their own programs. Today, millions of end users create numerous programs [Scaffidi et al, 2005] And they make a mess out of it  Christopher Scaffidi, Mary Shaw, Brad A. Myers: Estimating the Numbers of End Users and End User Programmers. VL/HCC 2005: 207-214

  3. Agenda • Motivation/Definitions • Examples (of EUP toolkits) • Alice, SUIT, Garnet, HANDS, SWiFT, … • Historical trends • Key techniques • VPL, PBD, Natural Programming, Application specific languages,… • Challenges

  4. Definitions • “Programming” • ‘‘The process of transforming a mental plan of desired actions for a computer into a representation that can be understood by the computer’’– Jean-Michel Hoc and Anh Nguyen-Xuan • “End-User Programming” • Programming by people who write programs, but not as their primary job function. • Instead, they must write programs in support of achieving their main goal, which is something else. • Covers a wide range of programming expertise • Business analysts, Neuroscientists, Physicists, Teachers, Accountants, etc.

  5. Definitions • “End User Programming toolkits” • An assembly of tools; set of basic building units that help end users do end user programming, mostly by manipulating or creating graphical user interfaces. • Examples: • Toolkits for: • Learning to write code • Creating web pages • Simulations • Email filtering • Other user-created functionality.

  6. Major Techniques • “Visual Programming” • “Programming in which more than one dimension* is used to convey semantics.” - Myers, 1990 • A programming language that lets users create programs by manipulating program elements graphically rather than by specifying them textually - Golin , 1990 • “Programming by Demonstration” • An end user development technique for teaching a computer or a robot new behaviors by demonstrating the task to transfer directly instead of programming it through machine commands. • Dan Halbert, 1984 • “NaturalProgramming” • Set of techniques to make making programming languages and environments easier to learn, more effective, and less error prone. - Myers, 1990- * Multi dimensional graphical objects, spatial relationships, time relationships (before/after)

  7. Example ToolkitsAlice [Randy Pausch, 2000] Youtube Video

  8. Example ToolkitsGarnet [Myers, 1988] • One of the earliest GUI development toolkit • Aimed at tools to help designers create, modify and maintain highly interactive, graphical, direct manipulation interfaces.

  9. Example ToolkitsSUIT [Randy Pausch, 1992] • Provides a collection of screen objects described by its: • state • a C procedure that displays it • a C procedure that handles input and updates the object state • Similar toolkits : UIMX, Interviews, Motif widgets Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344.

  10. Example ToolkitsHANDS [J.F. Pane, 2002] J.F. Pane, "A Programming System for Children that is Designed for Usability," Ph.D. Thesis, Carnegie Mellon

  11. Example ToolkitsOzone – a publish-subscribe based widget composition and creation toolkit • A popular framework that enables rapid creation, assembly and configuration of rich, lightweight widgets. • Promoted to create a common widget repository for the US military – similar to ‘Android Market’. Link

  12. Example ToolkitsRule based programming (Agent Sheet, MOSL, VISPATCH, SAM, VXT, …) • Programming Paradigm for Programming with Pictures, particularly visual Simulations and rules. • Rule based design to: • animate agents • play sounds and MIDI instruments • speak text • react to mouse and keyboard events • visualize values as colors and plots • query web pages, control web browsers Agent Sheet: http://www.agentsheets.com

  13. Example ToolkitsDataflow based (PROGRAPH, ObjectFlow, The Cube Language, SWiFT,…) • General-purpose Data-flow driven languages • Information flows through components • Data is transformed at certain points in the program Method case Operations

  14. /usr/local/fsl/bin/flirt -ref standard -in example_func -out example_func2standard -omat example_func2standard.mat -cost corratio -dof 12 -searchrx -90 90 -searchry -90 90 -searchrz -90 90 -interp trilinear Example ToolkitsSWiFT– a web-based toolkit web service composition (Neuroscience example) Align A large script file that contains program calls Spatial filtering Temporal filtering Program (a large number of binaries that perform one or more functions) Parameters (numbers range from 5 to 25)

  15. Historical trends • Action Graphics • FORMAL • ThingLab • Hi-Visual • LabView • PROGRAPH • PIGS • Pict • Rehearsal • SmallStar • Forms • Editing by Example • PICT • Lotus 1-2-3 • SIL-ICON • VisiCalc • HiGraphs • Miro • StateMaster • AMBIT/G/L • Grail • GAL • Graphical Program Editor • Query by Example • Pygmalion • I/O Pairs • Cube • Cantata • SchemePaint • CODE 2.0 • Iconicode • MViews • LOFI/HIPI • FOXQ • VMQL • GXL • Euler View • Yahoo Pipes • Popfly • AVS • Mondrian • ChemTrains • Vampire • VIPR • SPE 1980 1960 1990 2000 • Techniques/Goals • 3D Rendering • Visual Hierarchy • Procedures • Control Structures • Programmable Graphics • Animations • Video Imagery Exploitation • General purpose, declarative language • Audio, video and image processing • Graphical models from behavioral models • Learning and Cognitive abilities in vision processes • Handling Scalability, typing, and imperative design • Collaborative Software Development • Techniques/Goals • Child Learning • Xqueryby FORMS • Spreadsheet Analysis • Visual Model Query • Layouts • Specification and Interchange • Mashups • Web-based design • Programming for end-users (non-Professionals • Techniques • Graphs • Flowcharts • Flowchart derivatives • FORMS • Demonstrational • Techniques • Graphs • Flowcharts • Flowchart derivatives • FORMS • Demonstrational • Data Flows • Spreadsheets • Matrices • Jigsaw Puzzles • Petri nets • Flowchart derivatives

  16. Historical trends • Let users program in Visual Languages • (Almost) Make textual languages redundant • Make programming more accessible • Natural programming • Support domain -specific designs • Strive for improvements in programming language design • Support the cognition aspect of Programming 1960 1980 1990 2000

  17. Key TechniquesProgramming by demonstration [Dan Halbert, 1984] • “Programming by Demonstration” • An end user development technique for teaching a computer or a robot new behaviors by demonstrating the task to transfer directly instead of programming it through machine commands. • Many toolkits based on this technique: • COCOA, HyperCard, Grammax, Mail Filtering Halbert, Dan (November 1984). Programming by Example. U.C. Berkeley (PhD dissertation)

  18. Key TechniquesVisual Programming by demonstration • “Visual Programming” • “Programming in which more than one dimension* is used to convey semantics.” - Myers, 1990 • Many, many toolkits based on this technique. • Myers’ Taxonomy - Myers, 1990 • Burnett’s Taxonomy - Burnett, 1994 Brad Myers, Taxonomies of Visual Programming and Program Visualization, 1990 Margaret Burnett, A Classification System for Visual Programming Languages, 1994.

  19. Key TechniquesNatural Programming • “NaturalProgramming” • Set of techniques to make making programming languages and environments easier to learn, more effective, and less error prone. - Myers, 1990 - Present • “Brad Myers Natural Programming project” • http://www.cs.cmu.edu/~NatProg/index.html

  20. Key TechniquesDirect Manipulation • “Direct Manipulation” • A technique that involves users to directly manipulate objects presented to them, using actions that correspond at least loosely to the physical world. • “Central Ideas” • Visual Model of the world • Visual objects that can be operated on • Results of actions are reflected in the objects immediately. • Objects, once operated on, can be further operated on. - Shneiderman, 1983 Shneiderman, Ben. "Direct Manipulation. A Step Beyond Programming Languages" (HTML). IEEE Transactions on Computers

  21. Key TechniquesApplication/Domain specific Languages • “Application/Domain specific Languages” • A popular trend is to provide toolkits that allow end users to create domain/application specific platforms. • Many of them are VPL platforms • Examples: • Dataflow based: Yahoo Pipes • Pub-sub based (widget composition): Ozone, Synapse • Service composition based: Loni Pipeline, Taverna • Ontology based: WINGS • Scripting based: TCL-TK, various domain-specific scripts

  22. ChallengesUnderstanding the end user ecosystem Scientists, Physicists, Astronomists Accountants, Moms and Pops People with Professional end user software developer role [Segal, 07] People with a (technically) novice role Domains that involve writing a lot of code Domains that involve adaptations of software, and change in configurations Domains that involve using or adapting turn-key software • The goal of End user programming is to support their professional task. • But it is not always that they perform all these roles themselves.

  23. ChallengesUnderstanding the end user ecosystem… • “Multiple roles in Component based software development” • Framework builders, Component developers, Application assemblers, and end users • Additional role: end user developers – mainly involved with tailoring the applications. • Successful tools like Taverna, WINGS, LONI-pipeline are built on such ecosystems. Morch, A., Stevens, G., et al. (2004) Component-based technologies for end-user development. Communications of the ACM, 47(9), pp. 59-62

  24. ChallengesReuse is hard • Supporting reuse of existing end user code requires efficient packaging and search. • Current work*: • Package new capability as plugins [Scaffidi 2008, Sestoff 2002] • Finding resources through hierarchy of repositories from personal to local to global [Scaffidi, 2009] *mostly in context of spreadsheets

  25. ChallengesTesting, verification and analysis is even harder (and hardly available) • Most toolkits generate or select code as a result of user interaction. • Users typically have limited abilities to make changes in that code. • For example in the SUIT toolkit: • Creating custom widgets was hard • Teaching callbacks was not possible • Studies have demonstrated that programs produced using various end user programming toolkits are erroneous up to the level of 90% [Leventhal et al. 1994, Stolee et al. 2010] • There is limited support to analyze and fix such errors.

  26. ChallengesHigh Cost and low quality end user programming tools • Many end user programming tools are developed from scratch, leading to: • High cost of development, and • Low quality • Instead of many tools, what is needed is frameworks to support: • Component level reuse • Testing and analysis • Execution support, and finally • Packaging and sharing

  27. Next… We discuss an approach called End User Architecting that is aimed to address some of these challenges…

  28. References • Agent Sheet: http://www.agentsheets.com • Ben Shneiderman. "Direct Manipulation. A Step Beyond Programming Languages" (HTML). IEEE Transactions on Computers • Brad A. Myers, John F. Pane and Andy Ko, "Natural Programming Languages and Environments". Communications of the ACM. 47(9), pp. 47-52. • Brad A. Myers. "Taxonomies of Visual Programming and Program Visualization," Journal of Visual Languages and Computing. vol. 1, no. 1. March, 1990. pp. 97-123. • Christopher Scaffidi, Christopher Bogart, Margaret M. Burnett, Allen Cypher, Brad A. Myers, Mary Shaw: Predicting reuse of end-user web macro scripts. VL/HCC 2009: 93-100 • Conway, M., Audia, S., Burnette, T., Cosgrove, D., Christiansen, K., Deline, R., et al. (2000, Apr 1-6). Alice: Lessons Learned from Building a 3D System For Novices. Paper presented at the Proceedings of • CHI 2000, The Hague, The Netherlands. • David Garlan, Bradley Schmerl, Vishal Dwivedi, Aparup Banerjee, Laura Glendenning, Mai Nakayama, and Nina Patel. Swift: A tool for constructing workflows for dynamic network analysis. http://acme.able.cs.cmu.edu/pubs/show.php?id=333, 2011. • F.H. Post and W. Barth, Construction Techniques of Graphic, Direct-Manipulation User Interfaces, EUROGRAPHICS ’91 / Elsevier Science Publishers B.V. Eurographics Association, 1991.

  29. References… • Halbert, Dan (November 1984). Programming by Example. U.C. Berkeley (PhD dissertation) • J.F. Pane, "A Programming System for Children that is Designed for Usability," Ph.D. Thesis, Carnegie Mellon • Joseph Lawrance, Steven Clarke, Margaret M. Burnett, Gregg Rothermel: How Well Do Professional Developers Test with Code Coverage Visualizations? An Empirical Study. VL/HCC 2005: 53-60 • K. T. Stolee and S. Elbaum, "Refactoring Pipe-like Mashups for End User Programmers," International Conference on Software Engineering (ICSE), Honolulu, Hawaii, May 2011. to appear. • Margaret M. Burnett, “Visual Programming” In the Encyclopedia of Electrical and Electronics Engineering (John G. Webster, ed.), 1999 • Morch, A., Stevens, G., et al. (2004) Component-based technologies for end-user development. Communications of the ACM, 47(9), pp. 59-62. • Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344. ACM DL Ref • Scaffidi C., Myers B.A., AND Shaw M. 2008. Topes: Reusable abstractions for validating data. International

More Related