Electronic Abalone Referee

1. Electronic Abalone Referee By Benjamin Bayes (EE, 2005) & Dean Miller (EE, 2005)Advisor: Prof. Rudko • Enjoy the full gaming pleasure without dispute

2. In this presentation • Introduction to the game “Abalone” • Different options that could have been used for image acquisition for this project • Various MATLAB commands used to process the image • A demonstration of the game and of the designed referee system

3. Introduction • Abalone is a board game involving marbles that is based heavily on strategy. • The object is to push 6 of your opponents off of the board before it happens to you

4. Rules of The Game • Starting Position • Broadside Move • In Line Move

5. Rules of The Game • Sumito (Pushing) • Impossible Moves • Ejecting

6. Rule Variations • Alternate Starting Positions • Rules for 3-6 Players

7. Project Objective • To make a digital referee for an unaltered Abalone • Referee would sense every move made • Would identify legal moves, and flag illegal ones • Allow the players to reset the game if move is illegal • Would display score, moves, and a digital representation of the board • Compatible with 2 players (due to time restraints, we were unable to expand it to a possible 6 players)

8. Previous Implementations • There are many digital versions of Abalone available • The Abalone company offers a free online multiplayer computer version on their website • There are also many freeware and shareware single player and multiplayer versions • But no digital interface between a board and a computer could be found

9. Quick fact about the game • Typical game has only black and white marbles, but rule variations allow for up to 6 different colors (the colors chosen for this project were white and red- to be explained later) • Using image processing in MATLAB, the marbles and open spaces can be detected

10. Different Image Acquisition Options • We considered many different types of sensors and other data gathering devices such as: pressure and light reflecting sensors, camera sensors, IR sensors, etc.

11. Photoelectric Sensors • Ideal for differentiating between colors • Small and compact enough to fit into the holes of the game board • Come in different complexities • Unfortunately, not possible to cost and possible reliability issues

12. Pressure Sensors • Use a silicone top combined with a hydraulic system to help determine when pressure changes • Connected by pins on the other side of sensor • Unfortunately, there would be too much construction (hardware) and cost for this method

13. IR and Light Reflecting Sensors • These sensors are good in determining different colors • However, they are unable to tell exact location • Again, cost and complexity are issues

14. Digital Cameras and Webcams • A very common technology • Could be mounted above the board at a fixed position, or at a variable position • Cheap • Would rely on image processing in MATLAB

15. Choosing a webcam • Cheapest device (~$25!) • Pictures can be loaded into MATLAB and analyzed • Resolution is sufficient to determine location and the color of a marble

16. Layout and set up of the board • For ideal color and location detection, the color underneath the holes was made black • The marble colors being used in a two-player game are white and red

17. MATLAB CODE • Approaches Operator independent, automatic detection of marbles and holes based on edge detection, histograms, thresholding, conversion from color to black and white, separation of images into red, green, and blue components

18. Examples of MATLAB Experimentation • Initial image in MATLAB (color) • Conversion of color to Black and White • Edge detection (from Black and White scale) Not shown: histograms, and separation of color components

19. Problems Encountered • Edge Detection: Works well in detecting different colors (i.e. white on black) but does a poor job detecting different levels of black. • Similarity of colors: Need to enhance the images so that the differences in color are more distinct. • All Image Processing Tools: Noise problems are very difficult to solve! No matter what tools used, it will always be very difficult to isolate individual parts of an image!

20. Image Processing Solution • User initialized board • Once endpoints are known, all other points can be found through midpoint equations • Processing based on the analysis of the RGB values at each point

21. Detection • After each point is found, image RGB components are thresholded to determine marble color and space location • On a 0-255 scale, White has high values for R, G, and B, Red has high values of R but not of G and B, and Black has low values for R, G, and B

22. Analysis of Moves • Analysis of moves is based on a logic system • There are four kinds of moves (push, slide, summito, ejection) and multiple directions to execute each move • Ex. A 2 marble slide could be executed in 6 different directions • Logic System should check for all possibilities, and check for all possible errors

23. Logic System • Given: Board’s old and new position (lbig: 11:21:2 dimensional array) created from image processing • # Red, # White marbles on old and new boards (NR1,NR2, NW1, NW2 from lbig) • # of changes and change coordinates (change, ccoor(2:X array), from lbig) • How do we determine legal moves? • Solution: 1000+ lines of If, ElseIf statements!

24. lbig(:,:,1) = • W W W W W • W W W W W W • W W W • R R R • R R R R R R • R R R R R • lbig(:,:,2) = • W W W W W • W W W W W W • W W W • R R R • R R R R R R • R R R R R

25. %Now check for ejection • %Check for <--- and ---> direction ejection • elseif ccoor(1,1) == ccoor(1,2), • %Check for 2 - 1 ejection • if (abs(ccoor(2,1) - ccoor(2,2)) == 2 & lbig(ccoor(1,1), ccoor(2,1), move) == '') |... • (lbig(ccoor(1,2), ccoor(2,2), move + 1) == '' | abs(ccoor(2,1) - ccoor(2,2)) == 4), • if lbig(ccoor(1,1), ccoor(2,1), move) == lbig(ccoor(1,1), ccoor(2,1)+ 2, move) & ... • (lbig(ccoor(1,1),ccoor(2,1), move)== 'W' & lbig(ccoor(1,2),ccoor(2,2), move)== 'R' | ... • lbig(ccoor(1,1),ccoor(2,1), move)== 'R' & lbig(ccoor(1,2),ccoor(2,2), move)== 'W') ... • | ... • lbig(ccoor(1,1), ccoor(2,1), move + 1) == lbig(ccoor(1,1), ccoor(2,1)+ 2, move + 1) & ... • (lbig(ccoor(1,1),ccoor(2,1), move + 1)== 'W' & lbig(ccoor(1,1),ccoor(2,1), move)== 'R' | ... • lbig(ccoor(1,1),ccoor(2,1), move + 1)== 'R' & lbig(ccoor(1,1),ccoor(2,1), move)== 'W'), • disp('A legal 2 - 1 ejection has occurred.'); • delete (files1(1).name); • if NW1 - NW2 == 1, • scoreR = scoreR + 1; • disp('Red has scored.') • else • scoreW = scoreW + 1; • disp('White has scored.') • end • else • disp('Er.11 Marbles have moved too far. Please reset the board.') • delete (files2(2).name); • lbig(:,:,1) • if turn == 0, • turn = 1; • else • turn = 0; • end

26. Results • Program works! Can referee a complete game • Continuous image creation through Creative Webcam Software and camera mounted to stand • Program initializes board, keeps track of whose turn it is, keeps track of score, determines the winner • If an error is made, program displays last correct board orientation and waits for user to correct • Compatible with multiple staring positions • However, did not have time to make program compatible for multiple players/colors

27. Future Work • Could be adapted for 3-6 players and 3-6 colors • Further bug testing • Could implement an artificial intelligence program to compete against human opponents (A possible new senior project?)

28. Bibliography • Many meetings with Professor Rudko • MATLAB, “Image Processing Toolbox”- user’s guide. The MathWorks Inc, 1997 • “Computer Vision” Linda Shapiro and George Stockman. Prentice-Hall Inc, 2001 • http://www.sensortechnics.com/index.php?fid=300&fpar=YToxOntzOjQ6InBjaWQiO3M6MjoiNzIiO30%3D&isSSL=0&aps=0&blub=15a908b8f3eb6ba5eb42abd268d3faa5

29. Any Questions? Special thanks to Professor Rudko for his help!