Human-Computer Interaction Laboratory Interdisciplinary research community

1. Creativity Support Tools:A Grand Challenge for Interface DesignersBen Shneiderman (ben@cs.umd.edu)Founding Director (1983-2000), Human-Computer Interaction LaboratoryProfessor, Department of Computer ScienceMember, Institute for Advanced Computer Studies & Institute for Systems ResearchUniversity of MarylandCollege Park, MD 20742UIST Vancouver November 3, 2003

2. Human-Computer Interaction Laboratory Interdisciplinary research community - Computer Science & Psychology - Information Studies & Education www.cs.umd.edu/hcil

3. User Interface Design Goals • Cognitively comprehensible: Consistent, predictable & controllable • Affectively acceptable: Mastery, satisfaction & responsibility Design philosophy: Direct Manipulation NOT: Adaptive, autonomous & anthropomorphic

4. Scientific Approach(beyond user friendly) • Specify users and tasks • Predict and measure • time to learn • speed of performance • rate of human errors • human retention over time • Assess subjective satisfaction(Questionnaire for User Interface Satisfaction 7.0) • Accommodate individual differences • Consider social, organizational & cultural context

5. Scientific Approach(beyond user friendly) • Specify users and tasks • Predict and measure • time to learn • speed of performance • rate of human errors • human retention over time • Assess subjective satisfaction(Questionnaire for User Interface Satisfaction 7.0) • Accommodate individual differences • Consider social, organizational & cultural context 4th Edition: April 2004 www.awl.com/DTUI

6. Library of Congress • Scholars, Journalists, Citizens • Teachers, Students

7. Visible Human Explorer (NLM) • Doctors • Surgeons • Researchers • Students

8. NASA Environmental Data • Scientists • Farmers • Land planners • Students

9. Bureau of Census • Economists, Policy makers, Journalists • Teachers, Students

10. NSF Digital Government Initiative • Find what you need • Understand what you FindUMd & UNC www.ils.unc.edu/govstat/

11. Creativity Support Tools

12. Creativity Support Tools

13. Creativity Support Tools

14. Creativity Support Tools: Goals • More people, more creative, more of the time • Revolutionary breakthroughs, paradigm shifts, H-creativity • Evolutionary, normal science, music & art, creative knowledge work • Impromptu everyday creativity • Raised expectations for professionals • Tailored solutions • Cheaper, faster, better

15. Structuralists: A plan, method, process • Polya's four steps in How to Solve It (1957): • 1) Understanding the problem • 2) Devising a plan • 3) Carrying out the plan • 4) Looking back • Couger (1996) reviews 22 "creative problem solving methodologies" • Preparation • Incubation • Illumination • Verification

16. Structuralists: A plan, method, process • Atman's design steps: • Problem definition – identify need • Gather information • Generate ideas – brainstorm & list alternatives • Modeling – describe how to build • Feasibility Analysis • Evaluation – compare alternatives • Decision – select one solution • Communication – write or present to others • Implementation (Atman et al., Design Thinking Research Symposium2003)

17. Inspirationalists: Aha, Aha, Aha! • Free associations • Brainstorming • Thesauri, photo collages • Random stimuli, inkblots • Breaking set • Getting away to different locations • Working on other problems • Meditating, sleeping, walking • Visualization • 2-d networks of ideas • Sketching

18. Situationalists: context, community, collaboration • Personal history • Family history, parents, siblings • Challenging teachers, inspirational mentors • Supportive peers and partners • Consultation • Peers and mentors • Early, middle and late stages • Information and empathic support • Motivations • Fame, legacy, admiration • Competition

19. Csikszentmihalyi’s book Creativity (1993) • 1) Domain: e.g. mathematics or biology "consists of a set of symbols, rules and procedures” • 2) Field: "the individuals who act as gatekeepers to the domain...decide whether a new idea, performance, or product should be included” • 3) Individual: creativity is "when a person... has a new idea or sees a new pattern, and when this novelty is selected by the appropriate field for inclusion in the relevant domain"

20. Genex (Generator of Excellence) Framework • Collect • Learn from previous works in digital libraries, the web, etc. • Relate • Consult with peers & mentors, early, mid & late stages • Create • Explore, discover, compose, evaluate possible solutions • Donate • Disseminate refined results and contribute to the digital libraries, the web, etc. (Codex, memex, genex: The pursuit of transformational technologies IJHCI, 1998)

21. Genex: Some potential down sides • Collect • Will knowledge of previous work limit imagination? • Relate • Could mentors discourage exotic ideas? • Could peers rip-off your innovation? • Create • Will using standard tools limit creativity? • Donate • Could the desire for intellectual property protection limit dissemination?

22. Eight Activities in Genex • 1) Searching & browsing digital libraries • 2) Consulting with peers & mentors • 3) Visualizing data & processes • 4) Thinking by free associations • 5) Exploring solutions - What if tools • 6) Composing artifacts & performances • 7) Reviewing & replaying session histories • 8) Disseminating results (Creating creativity: User interfaces for supporting innovation ACM TOCHI, 3/2000)

23. 1) Searching & Browsing Digital Libraries • Effective search: Basic Google Search • Improved multimedia search • Overviews & previews • Result set categorization & visualization • Multiple session searches (Clarifying Search, CACM 4/98)

24. 1) Advanced Search: Google

25. 1) Search: Overviews & Previews • Faceted search • Preview cardinality of result sets www.epicurious.com www.endeca.com

26. 2) Consulting with Peers & Mentors • Early, middle and late stages • Information and empathic support • Build trust by negotiated expectations • Email, listservs, newsgroups, discussion boards • Chat rooms, instant messaging, audio/video conferencing • Comprehensive online communities • Tele-medicine, tele-meeting, tele-democracy • Collaboratories

27. 2) Components of Negotiated Expectations • Clearly identify and refine through dialog • who I am • what I want to do • Declare understandings • Why I think you can help • How you can help (specific request with time period) • How much is in it for you (payment, shared honor, appreciation)

28. 2) Example Request for Consultation • Poor: Dear Prof. Shneiderman: Attached is my PhD proposal, please tell me what you think. • Better: Dear Prof. Shneiderman: I am a Computer Science PhD student at Imperial College. My advisor, Prof. Spence, knows you and I have read your papers. My work extends your concept of dynamic queries. I’ve attached my 2-page doctoral proposal in the hopes that you can give me your comments before I defend it in two weeks. Your input would be acknowledged and Dr. Spence would be glad to pay for your plane ticket to join the committee meeting.

29. 3) Visualizing Data & Processes • Visual bandwidth is enormous • Human perceptual skills are remarkable • Trend, cluster, gap, outlier... • Color, size, shape, proximity... • Human image storage is fast and vast • Opportunities • Spatial layouts & window coordination • Information visualization • Scientific visualization and simulation • Telepresence and augmented reality • Virtual Environments

30. 3) Visualizing Data & Processes • Visual bandwidth is enormous • Human perceptual skills are remarkable • Trend, cluster, gap, outlier... • Color, size, shape, proximity... • Human image storage is fast and vast • Opportunities • Spatial layouts & window coordination • Information visualization • Scientific visualization and simulation • Telepresence and augmented reality • Virtual Environments

32. Treemap Implementations

33. 3) TimeSearcher • Time series • Stocks • Diseases • Weather • Genes • User-specified patterns • Rapid search

34. 3) Information Visualization: Mantra • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand

35. 3) Information Visualization: Data Types • 1-D Linear Document Lens, SeeSoft, Info Mural, Value Bars • 2-D Map GIS, ArcView, PageMaker, Medical imagery • 3-D World CAD, Medical, Molecules, Architecture • Multi-Dim Parallel Coordinates, Spotfire, XGobi, Visage, Influence Explorer, TableLens, DEVise • Temporal Perspective Wall, LifeLines, Lifestreams, Project Managers, DataSpiral • Tree Cone/Cam/Hyperbolic, TreeBrowser, Treemap • Network Netmap, netViz, SeeNet, Butterfly, Multi-trees (Online Library of Information Visualization Environments) otal.umd.edu/Olive

36. 4) Thinking by Free Associations • Free associations • Brainstorming • Thesauri, photo collages • Random stimuli, inkblots • Breaking set • Getting away to different locations • Working on other problems • Meditating, sleeping, walking • Visualization • 2-d networks of ideas • Sketching

38. An Idea Visualization Tool The human vision is by far the most developed and powerful faculty. The Idea Processor exploits visual attributes such as: color, shape, size, scale, position, depth, link, icon, etc. Visual cues facilitate recall, association, and discovery. Diagrams and pictures help you to represent and solve complex problems. Visualization reinforces your short term memory. Towards Higher Abstractions Ideas and diagrams are the basic abstractions of the Axon Idea Processor. Ideas are shown as graphical objects and its relationship shown as links. You get the big picture at all times, and details can be hidden from view. Stimulate Recall & Creativity The Idea Processor has an integrated Checklist Management System and a library of Checklists on problem-solving strategies, words of wisdom, etc. Checklists are effective means of capturing and transferring knowledge, and it effectively amplifies your intelligence.

39. 5) Exploring Solutions - What If Tools • State space exploration • “Combinationist theory” • Spreadsheets • Simulation as a third paradigm of science • SimCity & Flight Simulator • Economic models • Weather forecasts

40. 5) Exploring Solutions - What If Tools • Terry & Mynatt: Previews

41. 6) Composing Artifacts & Performances • Initiate a new composition • Exemplars • Templates • Processes • Revise at multiple levels • Low • Middle • High • Evaluate and refine • Feedback about problems • Measurement (Composition, Hawaii Int’l Conf. on Systems Science, January 2000)

42. 6) Composing Artifacts & Performances • Initiate a new composition

43. 7) Reviewing & Replaying Session Histories • Record compact histories • Allow users to review & annotate history • Disseminate histories (histories as first class objects) • Send by email • Post to website • Consult synchronously & asynchronously • Edit, extract, combine, search • Replay: slow, fast, reverse • Macros to automate exploration (Photoshop) (Learning Histories, Computer Supported Collaborative Learning Conf., December 1999)

44. 7) Imaginary dialysis procedure exploration

45. 7) Adobe Photoshop History tools

46. 8) Disseminating Results • Disseminate refined solution to gatekeepers • Facilitate web publishing & focused advertising • Reach subscribers & organizational gatekeepers • Ensure quality by editors & reviewers • Contribute to digital library communities • Journals, books, resources, libraries, communities • Science fairs, student projects, galleries, performances

47. Genex: Integrated Framework for Software • Modular design to allow components • Common file formats to ease data movement • Consistent commands to reduce cognitive load • Shortcuts for experts & immediate feedback • Direct manipulation for rapid learning, powerful actions, low error rates, high retention

48. Genex Phases and Activities

49. Challenges for Creativity Researchers • Domain knowledge is vital • Creative work may take years • Individuals have highly varied approaches • Evaluation is difficult • Controlled studies are unrealistic • Case studies are not replicable • Theories are shallow

50. An Inspirational Muse: Leonardo da Vinci (1452-1519) Renaissance Man • Combined science & art • Integrated engineering & esthetics • Balanced technology advances & human values • Merged visionary & practical (MIT Press, Oct 2002)