Math-Puzzle: Equation Tutor for Sighted and Visually-Impaired Children

1. AAFG 2005 Math-Puzzle: Equation Tutor for Sighted and Visually-Impaired Children Jarno Jokinen Department of Computer Sciences University of Tampere Finland jarno.jokinen@uta.fi April, 2005

2. Math-Puzzle introduction • communication of mathematics is usually visual • formulas, diagrams, graphs etc. • it is very difficult for blind and partially sighted people / students to do mathematics and is one of the biggest obstacles for them in school and at the university

3. Math-Puzzle most of the work reported in developing techniques that deal with mathematics can be presented through next categories: • tactile as in Braille and other raised representations • haptic or forced feedback devices represent shapes and curves • tonal representing equations and graphs (sonification / audification) • audio aids that read equations with tools to help in reading process • integrated /multimodal approaches

4. Math-Puzzle http://www.learn4good.com/games/kids/double_digits.htm http://www.boowakwala.com/kids/math-game-kids.html Introduction into the games

5. Math-Puzzle http://www.gamealbum.com/keyword/math/

6. Math-Puzzle • project impetus • created a game instead of making tests with an existing game • mathematical game for blind children • after different ideas, Math-Puzzle was chosen • training in logics, memory and equations’ manipulation strategy

7. Math-Puzzle • the problems in question • is it possible to solve equations using only short speech cues? • what are the limitations? • what is the easy way for blind interaction to edit the equations in static or dynamical puzzle? (memory capacity or external memory aid see next slide) • how long does it take to solve a puzzle? • what are the parameters of the gameplay progress and player performance? • how much does it make difference when player has visual feedback or all the tasks (including navigation within the game-field) are presented through sounds? • how and in which order the equations could be solved? (strategies)

8. Math-Puzzle • different matrix approaches • static puzzle on the left • dynamic puzzle on the right

9. Math-Puzzle • game concept • 5x5 matrix with one equation in each row • equations are predefined and randomly picked for the matrix • the goal of the game is to change the places of the equation members so that all of the equations are true • the figures (only in the same column) can be swapped by clicking one number and then the other one • operators cannot be swapped as well • other controls are: • alt+R -> new game (reset) • alt+M -> minimize / maximize browser window • alt+S -> show / hide figures and operators • the puzzle completion time and the number of moves are calculated • the game is currently implemented only in www with limitations

10. Math-Puzzle • testing procedure • 5 technically aware adults (age ranged from 26 to 34) • 10 games per player with all three playing modes: • 1st visual, 2nd blind (hidden labels with sound cues), 3rd blindfolded • laptop pc with external mouse and headphones

11. Math-Puzzle • problems • sound feedback is not supported with mozilla browsers • completing the fourth equation usually completes also the fifth • if not, the answers are crossing each other in a way that makes it very difficult to solve the puzzle • the program sometimes gives 1 or more correct equations at the start which may also result in an error on the browser

12. Math-Puzzle • results and discussions • the game (blind mode) is hard for adults that can see • what about kids that can’t? They have better spatial understanding • some matrixes can be a lot faster solved than others – repetitions needed for a good estimate of skills • if you wan’t to hear the sound cues you need to be patient • sound is heard when the mouse is moved over the square, so if you want to hear it again you must move the mouse away and over again • completing the fourth equation is crucial as explained on next slide

13. Math-Puzzle OK OK OK OK about the fourth equation

14. Math-Puzzle • conclusions • game requires a lot of memorizing, but there are strategies that help • completing equations in some order • using mathematical rules (division and multiplication) • when playing the game blindfolded, the increase in speed was bigger than when playing the visual version

15. Math-Puzzle References http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm http://www.gamealbum.com/keyword/math/ Children’s math project http://www.udel.edu/educ/cmp2/ http://www.educational-software-directory.net/math/ Lambda-project: Linear Access to Mathematic for Braille Device and Audio-synthesishttp://www.lambdaproject.org/ Karshmer, A.I., Gupta, G., Gillan, D. Architecting an Auditory Browser for Navigating Mathematical Expressions, ICCHP 2002, LNCS 2398, p. 477 http://link.springer.de/link/service/series/0558/papers/2398/23980477.pdf Gaura, P. REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002, LNCS 2398, p. 486, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdf Fitzpatrick D. Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical Material, ICCHP 2002, LNCS 2398, p. 494, http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf http://www.computing.dcu.ie/~dfitzpat/publications.html Math project, http://www.cs.york.ac.uk/maths/index.html Prosody in Mathtalkhttp://www.cs.york.ac.uk/maths/robert/prosody.html Mathematical Access for Technology and Science, http://www.papenmeier.de/reha/research/mathe.htm Edwards, A. D. N., Stevens, R. D. and Pitt, I. J. Représentation non visuelle des mathématiques, (translated by A. Assimacopoulos) in A. B. Safran and A. Assimacopoulos (editors) Le Déficit Visuel, Éditions Masson, pp. 169–178 (1995), http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.ps Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999