1 / 22

Multimodality

Multimodality. Dr. Charlotte van Hooijdonk LCC Meeting, 10 september 2008. Program. Multimodality Research done in my thesis Multimodality research within LCC context. What is multimodality?.

miyo
Download Presentation

Multimodality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimodality Dr. Charlotte van Hooijdonk LCC Meeting, 10 september 2008

  2. Program • Multimodality • Research done in my thesis • Multimodality research within LCC context

  3. What is multimodality? • A multimodal information presentation can be classified on the basis of three criteria or perspectives: • Delivery medium: devices used to deliver the information • Presentation mode: format of the message or the sign system used • Sensory mode: the human senses employed to process information

  4. Why is multimodality important? • Presenting information using multiple modalities is more effective than presenting information using a single modality (e.g., Mayer, 2001; Oviatt, 1999). • Moreover, recent developments in computer technology have led to new possibilities of presenting information and to a renewed interest in the effects of different presentation modes.

  5. Thesis: research questions Research questions • When and how do people present information in a multimodal way? • Which presentation modes are most effective for learning and executing procedural instructions?

  6. Thesis: research questions Research questions • When and how do people present information in a multimodal way? • Which presentation modes are most effective for learning and executing procedural instructions?

  7. Answer modalities How should my workspace be organized to prevent RSI? Monitor and keyboard should be in line with the user. Adjust the height of your chair in such a way that when you sit, the lower arms are level with the keyboard, without you pulling up your shoulders. The latter can be achieved easily by ensuring that the chair's armrests just reach the elbows when you sit with your shoulders in a relaxed position. Adjust the height of the chair in such a way that the armrests are level with or slightly above the top of the desk. Use the mouse and the keyboard without stretching your arms and/or bending your wrist. (www.unimaas.nl)

  8. Answer modalities How should my workspace be organized to prevent RSI? http://www.geckil.com/~harvest/rsi/

  9. Answer length What does RSI stand for? RSI stands for Repetitive Strain Injury. RSI stands for Repetitive Strain Injury. This disorder involves damage to muscles, tendons and nerves caused by overuse or misuse, and affects the hands,wrists, elbows, arms, shoulders, back, or neck.

  10. Research design • Participants • 111 Students from Tilburg University participated for course credits (46 male and 65 female). • Task • For 8 medical questions, provide two answers (one brief, one extended) -as you would like to have them- using a PowerPoint template with whatever (combination of) media you feel is best for a given question. • Coding system • In total 1775 answers were collected • Each answer was coded on the presence of visual media (and on the function of visual media.

  11. Results: functions of visual media • Decorative visuals did not add any information to the answer.

  12. Results: functions of visual media • Representational visuals represented parts of the textual answer. Their presence clarified the text.

  13. Results: functions of visual media • Informative visuals added information to the answer.

  14. Results: answer length

  15. Results: question type

  16. Thesis: research questions Research questions • When and how do people present information in a multimodal way? • Which presentation modes are most effective for learning and executing procedural instructions?

  17. Research design • We conducted an experiment studying a specific kind of procedural instructions, namely exercises for the prevention of RSI. • We varied the information modality (text versus static visual versus dynamic visual) and the difficulty degree of the exercises (easy versus difficult) into account. • Participants had to learn RSI exercises and were asked to execute them.

  18. Instruction in a text Place your right hand in front of your body with the elbow straight. Place your left hand on the palm of your right hand. Gently push the your right hand towards you until you feel a mild stretch.

  19. Instruction in a static visual

  20. Instruction in a dynamic visual filmclip.avi

  21. Results • None of the presentation formats was superior on all dimensions of interest, each had some disadvantages. • Certain RSI exercises seemed to be inherently easier to represent than others, which varied across different presentation formats.

  22. Multimodality research within LCC context • What? • Effectiveness of (combinations of) different presentation formats in different genres • Processing of (combinations of) different presentation formats • Adapting presentation formats to different user groups / needs • How? • Quantitative • Qualitative

More Related