A framework for designing Etiquette for Educational Technology

1. A framework for designing Etiquette for Educational Technology Punya Mishra & Kathryn Hershey<punya@msu.edu & hersheyk@msu.edu> Learning, Technology & Culture Program Michigan State University Etiquette in Human-Computer Work AAAI Fall Symposium November 2002

2. Thank you! … yes I mean you

3. People are suckers for flattery…… even for undeserved flattery(Cialdini, 1993) • People believe flattery • People like flatterers • Flattery is immune to validity • Criticism is not immune to validity • People prefer flattery to criticism

4. You can catch more flies with honey than with vinegar – anon.… but what does this mean for ed tech?

5. Will flattery get computers anywhere? (Fogg & Nass, 1999) • Simple computer game, akin to 20 questions, with feedback (no feedback, praise, criticism * with reason, arbitrary) • People believed and liked computers that flattered them • People didn’t care if flattery was valid • Criticism was only believed if it was valid • People disliked computers that criticized them, regardless of validity

6. Going beyond usability... A look back to flattery / criticism: Effect of praise/criticism depends on context (Attribution theory research: Meyer, 1982; Parsons et al., 1982; Nicholls & Miller, 1984; Graham & Barker, 1990) Perception matters: Praise/criticism related to perceived effort Effort/ability inversely related Praise given for success on an easy task has a negative effect on learner’s self-confidence Criticism of a poor performance can have positive effects on learner’s self-confidence

7. Essentially • Teacher expectations can be unintentionally communicated to students and can influence student achievement beliefs (Dusek 1985, Edmonds 1979, Graham 1991, Rosenthal & Jacobson 1968, Stipek 1993, 1996, Weary, et. al., 1989).

8. Research based on HHI work(Meyer, Mittag & Engler, 1986) • Pairs of participants worked on problem solving task and received feedback from “teachers” • Manipulations • Scored versus non-scored (i.e. ability level measured or not measured by the teachers) • Feedback • Praise on success in easy task + No blame on failure on difficult task versus • No praise on success in easy task + Blame for failure on difficult task • Each participant could see what feedback the other person received (though not the other person’s solution)

9. Results of HHI study • When teachers “knew” ability (scored condition) participants who received “No Praise for success & Blame for Failure” had a more positive evaluation of their own performance and greater positive affect • When ability was not known performance and affect did not differ with feedback

10. What we did (The HCI study) • Pairs of participants worked on computers solving task and received feedback from an evaluation machine • Manipulations • Scored versus non-scored (i.e. ability level measured or not measured by the teachers) • Feedback • Praise on success in easy task + No blame on failure on difficult task (Answer correct very impressive + Wrong answer. Task completed) versus • No praise on success in easy task + Blame for failure on difficult task (Answer correct. Task completed + Wrong answer, should have done better) • Each participant could see what feedback the other person received (though not the other person’s solution)

11. What we found (HCI case) • Significant main effect for Feedback F(1, 110) = 5.482, p =.05 • Same story held for comparison of their performance with the other student, positive affect, negative affect • No interactions were found significant

12. What does this mean… • Praise always won out • People ARE suckers for flattery • People DO respond socially to computers • But wait a minute… • Mediated life is not equal to real life • The Media Equation doesn’t apply across the board

13. The tension at the heart of etiquette • Obeying the social rules of the domain (curtsey) • Problem: Often social rules are implicit and embedded in cultures of practice. Or even when explicit they may be wrong! • Thoughtful consideration of others (tact) • Problem: It is often not clear what “others” are thinking/responding • Don’t have to conflict but they can and do…

14. Etiquette in edu tech • Adds another layer to this • Shift in focus from pleasure and ease of use to issues of learning and motivation etc. • Important part of learning is to be disturbing! • HCI is similar yet different from HHI?

15. Goals of learning/teaching • To know that (rote learning, conceptual understanding, performance) • To know how (strategies, activity, projects) • To have good self esteem • To enjoy, have fun • To inspire… These happen in many different contexts!

16. Etiquette: And Ed Tech • The goal • To understand the context(s) within which Educational Technology works • And how different contexts/instantiations of technologies for learning can impact design of ed tech • To bring existing research on teacher behavior etc. to the design of social technologies

17. Understanding Ed Tech thru Classification • Classification implicitly assumes values, beliefs, theories about technology, teaching & learning • Different classification schemes have ontological, epistemological and pedagogical implications (implicit commitments) • The nature of acceptable and desirable roles and relationships between learner and learning environment • Etiquette is dependent on how these roles and relationships are understood and constructed

18. Classifying ed tech • Alessi & Trollip (1991): focus on computer based instruction • Five categories: tutorials, drills, simulations, games, tests • Misses out on technologies such as wordprocessors, browsers etc. • Taylor (1980): Focus on roles played by computers • Three categories: Tutor, tool, or tutee

19. Classifying ed tech contd. • Means (1994): Moving towards pedagogical uses • Four categories: Computers can be used as a Tutor, used to explore, used as a tool and used to communicate • Bruce (2000): Based on Dewey’s impulses of the learner • Four categories: Media for inquiry, construction, communication & expression

20. The Chinese buffet approach… • Computer as tutor (drills, tutorials etc.) • Computer used to explore (simulations, games etc.) • Computer as a tool (for inquiry, construction, communication & expression) • Computer as a tutee (teachable software) • Computer as evaluator (Testing software)

21. Goals for ed tech etiquette • Broad roles of etiquette for HCI systems • (this morning) • Smooth interaction, sensitive to role, rank, task, criticality, context, culture • More specific goals for learning systems • Learner prior (and developing) knowledge • What it is that is to be learned • Learner motivation, affective issues etc.

22. Computer as tutor • Systems designed to teach by providing information, demonstration in a sequence determined by the system. Expository or practice • CAI, ITV, drills, tutorials etc. • Sage on a stage, didactic • System needs to indicate • It is more knowledgeable, pliable yet firm, sensitive to student’s knowledge state yet guiding towards right path

23. Computer to explore • Systems that facilitate student learning through exploration. Under student control, provides context for discovery • Simulations and games, virtual labs, microworlds etc. • Guide on the side • System needs to indicate • It is an arena for intellectual play, friendly, unobtrusive, that student mistakes are ok, should ask open-ended questions

24. Computers as a tool • General purpose tools for accomplishing tasks such as inquiry, composition, data manipulation, analysis, communication, construction, self-expression • Word processors, spreadsheets, presentation, web publishing systems, digital image/video processing systems… • The system should • Provide context neutral support and scaffolding, unobtrusive

25. Computer as a tutee • Systems that facilitate student learning through teaching the software. • Turtle Logo, Teachable agents project at Vanderbilt • The system should…

26. Computer as evaluator • Software specifically designed to evaluate student understanding, knowledge, or artifacts • Adaptive testing systems, Computerized GRE, LSA based techniques for scoring essays • System needs to indicate • Integrity, trustworthiness, consistency…

27. Women, fire & dangerous things … or the problems inherent in classification • Imposing rigid schemes on what are fuzzy categories with fuzzy boundaries • Simulation of frog dissection could be both a tutorial and an arena for exploration • A tutorial can also have an evaluative component • A web site can be a tutorial and a medium for communication or a site for exploration

28. Where do we begin? • Existing research on teacher behavior • Everything from pauses in speech to non-verbal behaviors have been studied • Learning from actual tutor-student interactions… • The Chris Miller two step • Place yourself in the place of the computer - what would you do? • Place a human in place of the computer - how would you respond to the person? • Nothing beats more research…

29. Just how messy can this be? Consider the case of unsolicited help • We know help is important and useful • However, consider the fact that research shows that students who receive unsolicited help are perceived to be of lower ability than students who did not receive help (Graham & Barker, 1990) i.e. if you need help you are low in ability (would you agree?) • So what happens when the computer tutorial or learning environment provides help? Solicited or unsolicited? We don’t know… yet. • It is an empirical question!

30. From etiquette to Ed-iquette • Need to understand contexts of activity • Need to look at existing research (teacher behavior and effects on student learning and motivation) • Yet be careful in adopting it • Two different strategies for integrating existing Ed Psy research • Using existing ed psy measures to evaluate interfaces (the case of accents) • Extend teacher behavior research into educational HCI (praise-blame)

31. One final thought • Should we anthropomorphize? • Maybe the goal is not to do so explicitly but rather as designers to become sensitive to how we can incorporate “etiquette” subtly…

32. You have been a great audience Thank you. Yes I do mean You!