Helping to bridge the gap between formal and informal learning : the HCI perspective

1. Helpingto bridge the gap between formal and informal learning: the HCI perspective Nelson Baloian Tokushima, Feb. 2011

2. Research goals • Discover meaningful novel learning scenarios for which • the “information loop” applies • Unified, coherent HCI across different platforms

3. Some “very old” Hystory • 1993 IPSI-GMD receives 2 of the first 8 electronic boards released outside XEROX.Palo Alto Labs. (LiveBoard) • Norbert Streitz PUBLISH: DOLPHIN • Ambiente: the disappearing computer • Ulrich Hoppe COSOFT: Cosoft • The Coputer-integrated Classroom • 1995 Move to Duisburg, COLLIDE

5. Back in Chile: Teaching Java Using “off-the-shelf” Software

6. Results of a survey • “The teacher simply spends too much time trying to show us things”, “We get easily distracted in class” • I started to take some books with me and began to read them there. Finally I quit attending classes”, • “Why does it take so much time to move from one program to another?”

7. TheCiC Project Duisburg 2001

8. TheJava Pallete in Freestyler

10. Lessons learned • HCI matters in learning scenarios • Minimize the interaction time • A lot of unexpected collaborative learning situations (as in Darmstadt) • Good for a particular situation -> the effort made by students to learn how to use it might not be really rewarding • “Set-up” time too long

11. DeepBoard • Henning Breuer, post-doc in DCC 2003, psychologist (HCI) suggests the development of a gesture-based e-board able to give some structure to the content • Flexible creation, structuring and presentation of learning materials, • Implementing “depth” to the interaction with interactive whiteboards. • Student access through multiple tablets and keyboards. • Web interface • Developed in Waseda by U. de Chile student Felipe Baytelman • Used for supporting video-based lectures in the GITS between Nishi-Waseda and Honjo

12. Lessons Learned • Keepit simple (HCI) • Simple structure • Simple HCI principles, easytoremember • Pages are nottheonly simple, easytorememberinformationstructure • 3D informationgraphs: nottoo complicatedtomanage • Stillbeingused !!

13. Mobile LearningScenario • 2005: cooperation with Gustavo Zurita • PhD thesis on mobile support for in-classroom learning (with M. Nussbaum) • Joint Idea: to use gesture-based interaction for mobile devices • Natural way to interact with a PDA • “Expand” the capabilities of the screen (deep board) • Other principles: • Keep rich face-to-face interaction while using computer technology • Dynamic group formation/reconfiguration

14. MCSketcher • Take a picture and start generating ideas on it collaborativeley • Interaction based exclusively on gestures, minimizing the number of widgets and the need of a virtual keyboard • maximizing the space available for content. • Content organized as a 3D concept map • No switching between sketching and gesturing

15. Darkened Margin denotes user is in an innersub-node Design spots show there are other sketching pages attached to this one, which can be explored by following the link shown as a yellow circle. In this case, there are 2 of them Highlighted Session menu shows the work needs to be saved The “group icon” shows that 2/3 of the users are in this node “Document three icon”

17. A frameworkfordevelopingmobileapplications • Many scenarios sharing certain common characteristics have similar requirements • Gesture recognition module • Able to recognize some gestures • Can be extended to add more gestures • A flexible, lightweight communication platform for peer-to-peer applications

18. The Middleware • Classes implementing an API for developing distributed P2P applications easily. • Available in Java and C# • Convertion of objects from internal representations into an XML representation, transmit them and convert the XML representation into the corresponding internal one. • Provides services for discovering partners in the ad-hoc network and establishing connections among the different applications in order to synchronize shared data.

19. TCP/IP connections to other applications for transmitting/ receiving objects Multicast UDP traffic for discovering and discarding partners TCP/IP connections manager Multicast discovery manager Active partners list Communication Node Discovering partners and establishing connection Functionality implemented in a single object which must be instantiated by the application in order to use the synchronizing functionalities

20. Application Application Communication node Communication node XMLSerializer Castor C# World Java World Sharing objects Data sharing mechanism based on a “shared objects”principle. A shared object is an abstract class which should be extended in order to create an object class whose state will be transmitted to all active participants when the object changes its state, this is when one or more variables change their value.

21. Example 1: Supporting rich interaction in the classroom with mobile devices(WMUTE 2008)

22. Mobile technology in theclassroom • Is it really necessary ? • Dinamic group reconfiguring during learning activity • Face-to-face interaction additional to shared workspaces • Personal gesture-based interaction allows for a flexible and swift exchange of information between teacher and learners

23. System Architecture View groups’ shared Working area Teacher Group 1 Send problems (re)configure groups Assesment Work together Group 2 Group 3

24. Group configuration Teacher’s module, in Group setup mode. Dragging user Nelson to group 2. Users are displayed automatically when discovered The groups are defined by the teacher. The icon will show allways What is actually on the groups’ shared workspaces

25. Problem creation writing the problem definition and the answers and delimiting the elements by closing them in rectangles For defining problems with Alternatives problem parts are Dragged into the respective areas

26. Synchronized work Screenshots of two students’ PDAs jointly working in the same group work solving different part of a problem

27. Sending the answers • Students must chose an answer by clicking on the “Share answer with group” button. (right) • After choosing an answer, the button turns into and agreement indicator (center). Here, the button shows the student’s answer coincides with one student but differs from two other. • Once all members agree on the answer, it turns into the “Submit answer” • button (right),

28. Assesment The teacher’ view in the “show results” mode. Group 1 has answered 2 problems right (one open and the other with alternatives) Group 2 has answered 2 problems with alternatives, one wrong one right and the other with alternatives Group 3 answered 3 problems 2 with alternatives (right) one open (wrong). Cheking of solutions in open problems is done “by hand”

29. Example 2: A Platform for Motivating Collaborative Learning Using Participatory Simulation Applications(CSCWD’07 )

30. Participatory simulations Role-playing activity oriented towards learning complex and dynamic systems Mapping real world problems to simulated context and behaviors Knowledge and patterns emerge from local interactions among users Highly effective in large groups Simple to set up and interact with Simple decision process: Analyze information, exchange information, make decisions and see the outcomes It allows to relate actions and their consequences Highly motivating even in large groups Participation and collaboration increase the understanding of the simulated reality and problem-solving abilities Mobility has positive effects in engagement Can be integrated in a whole classroom learning by doing

31. Designing (creating) roles &Items Example: A Trust building rules learningscenario

32. a b Designing Items Example: Diseases, symptoms and treatments

33. ExchangingItems: Proximity+ IrDA

34. ExchangingItems (Example:shares)

35. Teacher support to oversee the activity

36. Collaborative Learning of Cryptographic Protocols(with Prof. W. Luther) • In situations where cryptographic protocols are necessary there are various actors: Alice, Bob, intruder, authentication authority… • Students assume a role and trigger the corresponding action at the right time • Petri-nets used to check the correctness • First version: Duisburg 2006, for normal PCs

37. PCs interface screenshot

38. Problems-solutions • Participants need to synchronize face-to-face • Much better with mobile devices !!! • Flexibility to get together to discuss the sequence of the operations, reconfigure groups, etc.. • Last version: Students have to (re)configure the interface (place buttons in a meaningful position) Journal of Future Generation Computer Systems (available online)

39. Example 3: Mobile Collaborative Knowledge Management System(CSCWD’08)

40. support different processes of KM at the moment that is needed while the users are on the move use advanced information visualization techniques for displaying and navigating structures in complex information spaces Support face-to-face communication by interfaces that use gestures and sketching Organize knowledge using two-dimensional concept maps Contextualize the situation by proximity: IRDA functionalities embedded in mobile devices may be used to detect close proximity between 2 users wishing to exchange information. Design Principles of Mobile KM System

41. Description of the mobile KM System Creating nodes

42. Description of the mobile KM System Writing a label for a link

43. Description of the mobile KM System exchange of data by proximity with IRDA

44. Description of the mobile KM System The tree view

45. Description of the mobile KM System Contrasting coupled nodes. Green nodes belong to one user and blue ones to another. The coupled nodes are shown in the middle, one over the other. Doing a circle gesture over the coupled node will change the foreground-background order of the nodes view.

46. Formal evaluation of gestures • Which gesture is good on which platform ? • Diverse Platforms: PDA, smartphone, i-phone, i-pad, Galaxy-tablett, HP-Tablet PC, E-board, and we keep adding new ones ! • What is Good: easy to learn, easy to remember, robust (it does what the user wants to)

48. Towards unifying HCI principle across learning platforms • Fragmentation of learning experiences with different tools in different contexts create a need for integration of classroom activities (structured) & learning “in the wild” (unstructured) • Different interaction rules in each application • Briggs’ Technology Transition Model assumptions: “potential users will adapt technology if its benefits outweigh its disadvantages (notably, the cost of learning how to use it), adjusted by the frequency of use”.

49. Workflow: mobile → Individual→ Collaborative

50. Research Questions Is it possible to develop an integrated interaction model across platforms ? Widgets ? Gestures ? Both ? Which type of learning activities can be modelled in this way ? Can we describe them with a pattern language ?