CSCI 6363 Human Computer Interaction

1. CSCI 6363Human Computer Interaction Xerox Star, 1981 Class Introduction and User Interface History

2. Familiar Systems … • What is the below? • Right, a computer • Many (essentially all) courses concerned with, e.g., algorithms, architecture, operating systems, … • How many courses have you had concerned with these?

3. Familiar Systems … • What is the below? • Right, a human • Not so many courses in your curriculum talk about • However, without humans there would be no computer science!

4. Familiar Systems … • So, the subject(s) of this course – human computer interaction • Computer system, human, software, hardware, programming, models, … • And – most importantly, their interaction • Human-Computer Interaction: HCI

5. Familiar Systems … • So, the subject(s) of this course – human computer interaction • Computer system, human, software, hardware, programming, models, … • And – most importantly, their interaction • Human-Computer Interaction: HCI

6. Familiar Systems … • So, the subject(s) of this course – human computer interaction • Computer system, human, software, hardware, programming, models, … • Also, will be concerned with how to program display and interaction mechanisms common to user interface implementation

7. Familiar Systems … • So, the subject(s) of this course – human computer interaction • Computer system, human, software, hardware, programming, models, … • Also, will be concerned with how to program display and interaction mechanisms common to user interface implementation • Even learn some things about the human • Cognition, performance, …

8. Ever seen this?

9. Ever seen this? • What’s wrong here? • To start with, it doesn’t provide the user feedback about whether the last action was correct • It does not say it is the course schedule • Then, well, I guess it’s a matter of “selecting a term”

10. What’s a module? • Is it like a semester?

11. OK,scroll down • But, why do you (the user have to scroll down? • Why isn’t the usual, rather the exceptional, choice the one on top? • What is the user’s impression of the system and the organization using it? • It’s harder than it should be

12. OK,scroll down • Besides impression and satisfaction, what’s the cost? • About 3 seconds per user • If practiced • If no error • 3 secs * 3 times * 20,000 users = 50 hours • How long to do right? • ~ 0 hours

13. Why HCI and Interactive Systems? • The interface is all the user (customer, employee) sees! • Efficiency, success, satisfaction … • Practically … • In sw systems > 50% of code is ui code • And somebody has to program it • Discipline wise … • ACM SIGCHI is 2nd largest conference • SIGGRAPH in 10’s of k, SIGCHI k’s … many just hundreds • Bigger picture … • “External aids to cognition”, as critical for humans • Ability of computers to store and retrieve information may have deep implications • History of computing systems leads to emphasis on interactive systems • Context first

14. Overview of Interactive Systems • Humans interact with computers primarily through displays (and input devices) • In fact, lots of things going on that are part of the whole context of computing, humans and hci • Human • Interaction • Style, characterization, design • Programming systems • Other UTPA CS courses: • CSCI 6360: Graphics • CSCI 6361: Visualization

15. External Aids to Cognition

16. External Aids to Cognition • A tool is a tool is a tool … • (arguably) • Whether mechanical or electronic (information) • Levers multiply physical forces, information tools multiply … • Computer as “mind tool” … • Don Norman: • “The power of the unaided mind is highly overrated. Without external aids, memory, thought, and reasoning are all constrained. But human intelligence is highly flexible and adaptive, superb at inventing procedures and objects that overcome its own limits. The real powers come from devising external aids that enhance cognitive abilities. How have we increased memory, thought, and reasoning? By the invention of external aids: It is things that make us smart.” (Norman, 1993, p. 43)

17. Amplifying Cognition • Humans think by interleaving internal mental action with perceptual interaction with the world • This interleaving is how human intelligence is expanded • Within a task (by external aids) • Across generations (by passing on techniques and knowledge) • Computers in general, and interactive systems in particular, are important class of external aid

18. External Cognition – A Demo • In your head: 34 x 72 • Without special purpose algs, it’s a chore Need to remember intermediate results: 34 x 2 and 34 x 70 and add them • External aids make it easy to remember int. results • paper and pencil, easy • Or do away with altogether • calculator, fast

19. External Aids • In fact a long history of external aids • Slide rule exploits addition of logs for multiplication • Analog device • Computers are general purpose external aids

20. Ratio of Hu man Cost to Hardware Cost for Computing • Many years ago computers were large expensive unreliable machines • Von Neuman thought a few machines of capacity of Eniac would be enough for US • Plugging in cables, or setting switches, was reasonable use of operator time (both training and operation) • Cost of computer time relative to operator time was very high, • And tasks that were performed were relatively simple • A few calculations vs. interactive scientific visualization, or the human task of managing one's own "information space" ENIAC

21. Ratio of Human Cost to Hardware Cost • Interface styles have changed over time • As cost of hardware has decreased, has resulted in increasing need for efficiency of human time utilization

22. Ratio of Human Cost to Hardware Cost • Change in the relative cost of computer to human time, • coupled with tasks that can be performed due the processing power are responsible for the change in interface style • (put another way:) • Power of computer essentially ALLOWS more computing power to be directed toward the interface • Today, even "entry-level" PC's have "way more" processing power than needed for most task • e.g., word processing • Today, cost of hardware is essentially trivial compared to the cost of human user!

23. Ratio of Human Cost to Hardware Cost • Moore’s law is the 800 pound elephant in the room • But, no real change in change in interaction style since mid ’80’s • 25 year’s! • Approaching a complete human generation • Rather, more computing devices • Shneiderman notes and his current edition reflects this • Also, connectivity of devices and storage

24. Context of User Interfaces • Graphics is key enabling technology in evolution of computing environments: • Graphical user interfaces • Visual computing, • e.g., desktop publishing, scientific visualization, information visualization • Hardware revolution drives everything • ~ every 18 months, computer power improves by factor of 2 in price/performance • Moore's Law • … and exponential growth generally – things are different … • Graphics memory and network speeds are on even faster exponentials • graphics chips in particular have major improvements every six to nine months

25. Moore’s Law - CPUs • Hardware revolution drives everything • ~ every 18 months, transistor count improves by factor of 2 in price/count • Moore's Law

26. Moore’s Law - CPUs • Hardware revolution drives everything • ~ every 18 months, transistor count improves by factor of 2 in price/count • Moore's Law

27. Another Example of Exp. Growth • The Internet … by whatever metric

28. Another Example of Exp. Growth • And digital camera resolution ….

29. Taking Advantage of Exp. Growth?

30. On Exponential Growth … • Some “technology” is exponential in advancement • Other things are not … • E.g., software and …

31. Moore’s Law Dead? • … and transistor count not equal to power

32. Moore’s Law Dead? • Transistor count not equal to performance

33. Parallelism and Moore’s Law • Moore’s law is alive and well for graphics and may be fine too for “gpu computing” • Because lends itself to parallelization • Pipeline architecture works well with current approach to cg • Now, gpu processing power significant fraction of cpu processing power in pc’s • How to use parallel, here, gpu, architectures is not a new question • Cf. Nvidia gpu programming language (CL)

34. And Ratio of Human Cost to Hardware Cost for Computing … and “Neural Power” But, like, not … 1. Promise of future fallacy 2. Computer processing not (at all) like neural processing (but, cf. this month’s CACM, “Cognitive Computing”) Hans Moravec, When will computer hardware match the human brain? Dec. 1997http://www.frc.ri.cmu.edu/~hpm/

35. Ratio of Human Cost to Hardware Cost • Moore’s law is the 800 pound elephant in the room • But, no real change in change in interaction style since mid ’80’s • 25 year’s! • Approaching a complete human generation • Rather, more computing devices • Shneiderman notes and his current edition reflects this • Also, connectivity of devices and storage

36. Chronology of System Development(another take on style) Wise, 1997

37. W W W = Hypertext(a quick example of why vision and history matter)

38. W W W = Hypertext(a quick example of why vision and history matter) • Real “power” interest in Internet only arose well after its initial implementation • ARPANET • Relatively few people cared about ftp, bbs, … • At least “few” compared to current use of WWW • Interest and use arose only after: • Specification of WWW • How to go from one file/document to another via its address … the link! • And then only with design of a graphic user interface for it • Text-based browsers not too neat (at least to non-computer types) • How to “navigate” easily and with use of pictures/images/icons • Essential idea of “navigation” among documents dates to Vannevar Bush’s Memex, 1945

39. History: Vannevar Bush, MEMEX • MIT professor 25 years, etc. • Claude Shannon (information theory) was student • Roosevelt’s science advisor in WWII  • Invented continuous intergraph or Differential Analyses • Essentially, Charles Babbage’s Difference Engine with shafts driven by motors • Analog computing solutions to differential equations (gears, etc.) • Big and handmade

40. History: Bush's MEMEX, 1945 • "As We May Think", 1945 • MEMory EXtender system • Atlantic Monthly! • available at: http://www.theatlantic.com/atlantic/atlweb/flashbks/computer/bushf.htm • “new knowledge not reaching the people who would benefit from it” • Concerned about the explosion of scientific literature which made it impossible even for specialists to follow developments in a field, • “A Memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.” • desk, slanting screens, buttons, levers, and keyboard • “A memex looked like a desk with two pen-ready touch screen monitors and a scanner surface. It had several gigabytes of storage space filled with textual and graphic, indexed, information”

41. History: Bush's MEMEX, 1945 • Microfilm projectors for viewing different information • uses mircofilm for storage • new material can be added via microfilm or by direct entry via ‘‘dry photography'‘ • Supports indexing, cross­referencing, keywords • supports associative indexing via links and creation of ``trails'' which can later be followed • allows annotations ­ comments, and marginal notes . • envisions multimedia i/o: other senses, such as, speech and audio • Associative indexing • "The process of tying things together is the important thing.“ • New profession of ``trail blazing" • Trail building and trail following by user

42. Editor’s foreword to Bush’s 1945 Atlantic Monthly article As Director of the Office of Scientific Research and Development, Dr. Vannevar Bush has coordinated the activities of some six thousand leading American scientists in the application of science to warfare. In this significant article he holds up an incentive for scientists when the fighting has ceased. He urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge. For years inventions have extended man's physical powers rather than the powers of his mind. Trip hammers that multiply the fists, microscopes that sharpen the eye, and engines of destruction and detection are new results, but not the end results, of modern science. Now, says Dr. Bush, instruments are at hand which, if properly developed, will give man access to and command over the inherited knowledge of the ages. The perfection of these pacific instruments should be the first objective of our scientists as they emerge from their war work. Like Emerson's famous address of 1837 on "The American Scholar," this paper by Dr. Bush calls for a new relationship between thinking man and the sum of our knowledge. --THE EDITOR

43. Some Interactive Systems History

44. Some Interactive Systems History • Doug Englebart • A Turing Award for vision in interaction • Ivan Sutherland • A early 1960’s “paint program” that foreshadowed much • Alan Kay • The “personal computer” – also, dynabook, Apple II • Today’s ubiquitous interaction style • Xerox Star and Alto • Apple Lisa and Macintosh

45. Douglas Engelbart, 1963 • Turing Award, 1998 – this is a big deal • Augmentation of human intellect (1963) • “... increasing the capability of man to approach a complex problem situation, gain comprehension to suit his particular needs, and to derive solutions to problems.” • Recognized his ideas built on Bush's idea of a machine that would aid human cognition • hierarchical structures for ordinary documents • group creation and problem solving • NLS System (1965­1968): • outline editors for idea development • hypertext linking • tele-conferencing, word processing, e-mail • System required: • mouse pointing device for on-screen selection: Invented the mouse (1965) as a replacement for light pens for use in his NLS system • a one-hand chording device for keyboard entry • full windowing software environment • on-line help systems • concept of consistency in user interfaces

46. Sutherland’s Sketchpad • Ivan Sutherland • “Pioneer” of … lots of things • Visualization • Graphics • Interaction • Still around • Evans and Sutherland graphics • First truly interactive graphics system, Sketchpad • A fairly sophisticated “paint” (or drawing) program • MIT, Ivan Sutherland’s 1963 Ph.D. thesis • “Sketchpad, A Man-Machine Graphical Communication System” • Available: www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf • Video: www.youtube.com/watch?v=mOZqRJzE8xg • “Among most important works in computer science” Ivan Sutherland using Sketchpad in 1963 CRT monitor, light pen and function-key panel

47. Ivan Sutherland’s Sketchpad, 1963 • Regarded as the first to implement much of what called “visualization”, “immersion”, and “virtual reality” (not to mention cg) • Some quotes: • ….. If the task of the display is to serve as a looking-glass into the mathematical wonderland constructed in computer memory, it should serve as many senses as possible. • ….. By working with such displays of mathematical phenomena we can learn to know them as well as we know our own natural world. Such knowledge is the major promiseof computer displays. • ….. The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.

48. Sutherland’s 1960’s equipment • “Ultimate display”, 1965 • Sword of Damocles – 1st HMD • Actual camera-like metal shutters • Not OSHA approved

49. Advances in Computer Graphics for Interaction • Input devices such as data tablets (1964) • Display processors, capable of real­time image manipulation (1968) • Introduction of low­cost graphics terminals (1970s) • OS support for interactive graphics under timesharing systems • New programming languages for embedded graphics support • New applications flourished - computer­aided math libraries (1968) . - 3D computer graphics / molecular modeling (1968, 1977). - animation, movies, sequencing of images (1966, 1969)

50. The “Personal Computer”, 1 Dynabook (Alan Kay, 1977): Conceptualized notebook computer: cardboard prototype “...a self­contained knowledge manipulator in a portable package the size of an ordinary notebook...” powerful processor, lots of memory, high resolution graphics, high fidelity audio Altair 8800 (1975): Considered to be first microcomputer 2 MHz Intel 8080 with 256 bytes standard RAM and interfaced with the user through octal front panel switches appeared in Popular Electronics home computer people, could build for around $400 Altair 8800