Shuffleboard Scorekeeper Design Review

1. Shuffleboard ScorekeeperDesign Review Tim Myers Dan Stella Eric Beisecker

2. Outline • Project Overview • User Interface • GUI Design • Camera Cradle Design • Technologies • Run Time Operation • Scoring and Image Processing • Testing Strategy • Difficulties • Current Progress • Questions

3. Shuffleboard Scorekeeper – Overview • Scores shuffleboard rounds automatically. • Game score and statistics are displayed on an LCD Display. • The Scorekeeper keeps track of the action using two cameras connected to a PC to determine position of the pucks and their scoring value. • Players interface with the scorekeeper to start games and check high scores using a USB keypad. • Table-mountable system for use with multiple shuffleboard tables. Shuffleboard Scorekeeper Cutaway view with clamps

4. Shuffleboard Scorekeeper – Diagrams Shuffleboard Scorekeeper Side View Shuffleboard Scorekeeper Block Diagram Shuffleboard Scorekeeper Top View

5. User Interface • Users will interact with the Shuffleboard scorekeeper using a USB Keypad. • One keypad for each side of the table • Text entry is done using a “phone style” key combination. • Situational buttons ensure “scorekeeping errors” can be handled easily.

6. GUI - Design Problems • Users will be navigating the GUI with a keypad • Users will be standing five feet away from the screen

7. GUI - Solutions • Use a minimalistic design • Make selected object obvious, bright colors, large fonts • Keep GUI flow logical

8. GUI Design Flow Chart

9. Camera Cradle Design • Unobtrusive to game play • Sturdy for camera support • “Oversized cross beam” is bolted in place for flexibility

10. Technologies • AForge • C# based - Similar to java • Robust library of image processing classes • Includes video capture classes • Excellent demo programs • Free! • SQL • Keep track of users to remember scores • Easy C# interface • Transferability • Free!

11. Run Time Operation • Separate Tasks • Each side of the table has a camera • Cameras have different tasks depending on turn • Observing motion vs. scoring • Cameras switch tasks on turn change • Scoring Task • Observe changes in puck position • Score based on puck position • Motion Task • Detect puck “toss” • Signal scoring task to score current puck positions.

12. What’s behind the Scoring • Several different image processing techniques can be used to separate the playing field into components that can be represented separately. • Scoring Lines • Table Edges • Puck centers • Once each component set is available for observation the locations of each item can be compared to one another to determine relative location. • The relative location can then be used to determine how close each puck is to one another and where each puck is located on the overall playing field. • Before the game begins, an initial frame of the table will be taken and then processed in order to establish the scoring lines and table boundaries that will be used for scoring. This is the initial calibration and happens on startup. • A single frame will be taken from both cameras, one at a time, and then analyzed.

13. Clean ½ Table Shot

14. Establishing the Scoring Lines • The initial playfield frame will be processed with an edge detection algorithm. • The edge detection algorithm will show the edges of the table, the edges of the actual playfield, and the edges between each end of a scoring line and the table. • After the edge detection algorithm, a vertical line detection algorithm and a horizontal line detection algorithm is run on the new frame. • The horizontal lines will be the edges of the scoring lines, the actual playfield to the gutter, and gutter to the table wall, and the vertical lines the edges of the actual playfield to the gutter and the gutter to the table wall. • If using the knowledge that viewing the pixels from the center towards the edges, the first vertical lines reached are the gutter edges, the gutters can be located. • On top of this the boundaries of the scoring lines can be determined. Their length as well as their width(The area in between their respective edges) can be used to determine their slope and thickness.

15. Edge Detected Table Shot ScoringLine Center Gutter Edge

16. The Algorithm • The algorithm to determine bounds will use three different images. • The original color image • The horizontal line image • The vertical line image • All three images will share “coordinates” so a single pixel is essentially represented in three different ways. • Starting from the top and the center of the frame every column will be trolled for horizontal lines as long as the current pixel is not the color of the gutter (green) in the color image. • The columns will be searched until the current pixel is also a vertical pixel in the vertical line image. This means that the gutter end is reached. • All of the separate line data is then stored to have their slopes and “coordinate” locations kept for referencing later without needing to re-establish any boundary lines.

17. Three Shot Example

18. The Actual Scoring • Since the boundaries of the playing field and the scoring lines are kept. Scoring a zone is pretty simple. • Once notified by the motion task, the frame will be taken and color detected for both red and then blue to get the puck colors. These locations are then compared to the stored boundaries to get relative locations. • The puck closest to the bottom table edge will earn points for that team for every puck they have closer than the first puck of the opposite color. The locations of the pucks are compared to the scoring lines and the other pucks to determine the overall score. Red Color Detect Normal Frame Blue Color Detect Closest Puck Stored Edge lines

19. Testing Strategy • Unit Testing • NUnit • Write tests to ensure proper functioning of code • Easy detection of failure due to code changes • Integration Testing • Write user stories for different play scenarios • Maximum coverage of edge cases and possible player actions • Manual Execution

20. Difficulties • Image Processing • Puck Detection • Scoring • Turn Detection (Motion) • Hangers • Puck Collisions • “KnockOffs” • Cameras • Capture Speed • Resolution

21. Situational Difficulties • 1.) Differentiating colors • Reason: Could blur edge lines and possibly lose the edge of the table or scoring lines. • Solution: LED Lights, and carefully selected paint colors • 2.) Poor camera angle • Reason: Losing horizontal or vertical lines due to the angle they are at. • Solution: Tightly Fixed Cradles • 3.) Camera shaking/Table being knocked mid turn • Reason: The scoring lines won’t be calibrated every puck throw in order to save processing time. • Solution: Tightly Fixed Cradles, possible recalibration between rounds, and an option to recalibrate/rescore.

22. Current Progress

23. Current Progress

24. Future Enhancements • Database of Users • Keep track of player scores for multiple games • Additional Statistics • Number of wins, loses, playtime, average score, etc. • Playing Music • Play music during game play • Player Icons • Customized icons associated with player names • Tournament Support • Option to support elimination tournaments instead of single games • Additional Lighting • Add lights to camera cradle to ensure proper lighting

25. Questions?