1 / 39

Graphics and Words

Explore the use of visual displays and words in communication, the dual coding theory, the nature of visual and verbal languages, and the integration of images and words. Learn how to effectively use narrative threads and the power of shared environments in conveying ideas. Discover the benefits and limitations of visual and verbal communication.

parmer
Download Presentation

Graphics and Words

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Graphics and Words • When should we use a visual display? • When should be use words? • Dual coding theory • What is a visual language? – does it make sense? • How to integrate images and words • The Narrative thread

  2. Consider that hieroglyphs gave way to more abstract symbols • Why turn back the clock? -17K years

  3. Pavio’s dual coding theory Associate structure Imagens Visual Non-verbal responses Image Information Visual System Text Information Logogens Verbal responses Verbal Auditory System Information From speech

  4. Capacity of visual working memory (Vogal, Woodman, Luck, 2001) 1 second • Task – change detection • Can see 3.3 objects • Each object can have several attributes Orientation Shape size, texture

  5. Capacity of verbal working memory • Used to be though of a 7 +/- 2 • It is now thought of as more a duration of ~ 2sec and a store of proto-verbal codes. • Words have an enormously rich semantic structure • Sometimes we have a choice • Spatial prepositions: above, across, alongside, amidst, apart, around, back, behind, below, beneath, beside, between, beyond, by (out of approx 80: From Pinker)

  6. Reasoning with words and graphics • 1 2 3 4 5 6 7 8 9 • Get a set of numbers, such that 3 of them add to 15.

  7. Can we have a visual language?

  8. The nature of language • Words are arbitrary • Chomsky, grammar with innate deep structures. • Common to computer languages • Austin - Illocutionary force (speech acts) • Critical period for language development • But, being verbal is not essential to language development • Sign languages for the deaf are the most perfect examples of visual language

  9. Sign languages Danish SL Chinese SL ASL Arose spontaneously Are not related to verbal languages Have grammar Become more abstract over time NSL Goldin-Meadow

  10. What can we do with language? • Description • The ability to communicate procedures and sequences of operations – including logic – if, but, causes, do a then b then c • A vehicle for thought when internalized • Communicate goals, intentions (illucutionary acts). • Make things happen • Nit picking (mutual grooming)

  11. “If there is no fresh trout at Whole Foods then go to the fishmarket On 19th and get salmon.”

  12. There are special brain areas for language Production Production Broca Understanding Wernicke Auditory Cortex Visual Cortex

  13. Can we have a visual language?Yes – but …

  14. If we don’t learn it early somesome capabilities are best handled verbally

  15. The visual system gives us Rapid recognition and pattern finding

  16. Abstraction Pattern Org chart • Jane is Jim’s boss • Jim is Joe’s boss • Anne works for Jane • Mark works for Jim • Anne is Mary’s boss • Anne is Mike’s boss

  17. Visual and verbal pseudo-code • While letters in stack • Take a letter • Put a stamp on it • Put it in the ‘out tray’ Visual programming languages have a history of failure Data flow diagrams are defunct

  18. Examples of visual languages • Sanscrit • Petri-nets • Khoros

  19. Sum integers from 1 to 3 Sanscrit for i = 1 to 3 do sum = sum + i;

  20. Petri Net languagePetri nets are stochastic – timed attributed (tokens on nodes, transitions)

  21. Khoros

  22. Attaching images and words • Pointing, Diexis and the diectic gesture • Can be a glance or a nod • An elementary speech act • Shown to disambiguate verbal communications • Why the mouse is so powerful • The basis of shared environments

  23. Attaching words to images- use gestalt laws

  24. Integrated pictures and words moreEffective: Chandler and Sweller 1991

  25. Narrative: Visualizations can be used to help tell a story • Can something visual carry a narrative. • In psychological terms the important thing is a leading of the thread of ideas (=attention). • In data analysis the analyst has the narrative thread. • In a presentation the audience is following the presenter’s thread. (actually a hybrid is best).

  26. Narrative • Can be accomplished with either images or words • A leading of attention

  27. Degrees of narrative • Strong leading  Free exploration

  28. XPlane

  29. Recap • Words for procedure, logic and abstract meanings/ images for pattern and structure • Both words and images can sustain narratives (leading attention)

  30. Emotional and motivational effects (Images can be more efficient) and automatic.

  31. Gillnet Set Surface Buoy Lobster Trap Trawl Surface Buoy Landry 2002

  32. Theory: Graphics and Words • Words for procedural logic: conditionals, qualifiers, if-then else, while. + causality. • Not the formal logic of math but a kind of concrete logic. (A vowel must have an odd number on the back). 8 A 3 F

  33. Other kinds of gestures • Beat gestures for emphasis • Verb gestures showing how to do something • McNeil Hand and mind

  34. Issues in shared environments • Speech + Pointer + Visuals – most important components • Subtle ways of directing attention also important in meeting dynamics.

  35. Peter

More Related