Final Year Project 2004/2005 LYU0402 Augmented Reality Table for Interactive Card Games

1. Final Year Project 2004/2005LYU0402Augmented Reality Table for Interactive Card Games Supervisor: Prof Michael Lyu Presented by: Kevin Chow, Albert Lam

2. Overview • Introduction • Objective • Trading Card Game • Architecture of ART • Implementation in Second Term • Experiment • Conclusion • Future Work

3. Introduction • What is Augmented Reality? • Mixed Reality combines the content from the real world with virtual imaginary • Augmented Reality is a subset of this where virtual content is overlaid into real objects of the world

4. Mixed Reality (MR) RealEnvironment AugmentedReality (AR) Augmented Virtuality (AV) VirtualEnvironment Reality-Virtuality (RV) Continuum Introduction (Con’t) • Paul Milgram characterized Mixed Reality interfaces on his “Reality-Virtuality Continuum” in 1994

5. Introduction (Con’t) • An AR system has the following three main characteristics: • Combines real and virtual objects in a real environment • Runs interactively, and in real time • Registers virtual objects onto the real world

6. Introduction (Con’t) • ART stands for Augmented Reality Table • User can play Card Games on the ART. • ART can enhance the visual effect during the playing of the card game.

7. Objective • Develop a generic Interactive ART Trading Card Game application. • Enhance the traditional card games: • Visual effect • Sound effect • Complex calculation

8. Trading Card Game • Trading Card Games • Two players play a match. • Players play cards to fight with each other. It includes summoning monster, casting magic, or setting traps, etc. • We choose “YU-GI-OH” as our implementation

9. Trading Card Game (Con’t) • Trading card game includes five kinds of actions • Restore • Draw cards • Play cards • Attack/Challenge • Discard cards

10. An Example - YU-GI-OH • In YU-GI-OH, Different information are stored on the card.

11. An Example - YU-GI-OH (Con’t) • Game Mat of YU-GI-OH • Different kinds of card must be put inside particular card zones

12. An Example - YU-GI-OH (Con’t) 1200 – 500 = 700 -700 ATK : 1200 ATK : 500 LP : 2000 LP : 1300 LP : 2000

13. An Example - YU-GI-OH (Con’t) 1300 – 1200 = 100 -100 ATK : 1200 DEF : 1300 LP : 1300 LP : 2000 LP : 1900

14. Architecture • Hardware Setup • Plasma monitor • Overhead camera • System architecture • Augmented Reality Perception • Game Core • Database • Game Enhancement

15. Hardware Setup

16. Hardware Setup (Con’t) • Computer • processes perceiving image • generates visual and audio enhancement • controls game flow • Plasma Monitor. • acts as the Game Table • displays computer-generated scene • Overhead mounted camera • captures the cards and the screen of the plasma. • only input of the system

17. System Architecture • Game Core Module • Perception Module • Database Module • Game Enhancement Module

18. System Architecture (Con’t)

19. Implementation • Database Module • Access card information • Access game rules and card rules • Identify cards • GameCore Module • Control game flow • Store game state • Rulebase game engine

20. Database Module • Contains all game information • Card Database • Contain all card information • Simple editor • Rule Database • Contain all game rule and card rule • Simple editor

21. Image Database • Image Database • Query with a card image • Retrieve card id • Use improved block matching algorithm ?

22. Block Matching Algorithm • Select the best matched card from the candidates • Split the captured image into 4 channels • Compare only inner image • Compare the different channels to the database separately

23. Block Matching Algorithm • Find the pixel difference by • Reject the image if the pixel difference is larger than the threshold • Accept the image with minimum pixel difference

24. Improved Block Matching Algorithm • However, the Block Matching Algorithm cause some error in real time game • Solution, we use an improved Block Matching Algorithm • Divide the image into 9 squares • Apply Block Matching Algorithm to each squares

25. GameCore • Core Part of ArtTable • Contains: • Input Manager – handles input • Output Manager – handles output • Rule Manager – stores rules and inference rules • Game Manager – stores game states • Control how these managers interact and flow of the game.

26. GameCore 1. Receive input 7. Display output to player Input Manager Output Manager GameCore 2. Pass to game manager 6. Generate output if any Game Manager Rule Manager 5. Perform actions if any 3. Update game states 4. Inference rules

27. Rulebase • Why rulebase? • Flexible – Card games are very flexible and some card effect are unpredictable. • Extensible – New cards can be extended by adding new rules. • Generic – New card games can be implemented by new rule models.

28. Predicate1 Predicate2 Conclusion ActionList Rulebase • Load game rules • Forward checking for predicate • While conclusion reached • Perform the action of the rule • further forward checking P.S. Action may be load another rule, remove a rule, reset a rule, update game states, or show some outputs, etc.

29. MainPhase PutCard1 NormalSummon MainPhase PutCard2 PlaySpell HandFromHell PlaySpell SetTarget DestroyMonster MysticTomato SendGraveYard PutCard1 SpecialSummon Rulebase

30. Experiment • We have used 90 cards to do the recognition experiment. The result is as follow:

31. Experimental Result Discussion • Reasons decrease the accuracy • Image are captured in low resolution. The small error in cards position detection will lead to inaccuracy in recognition process. • Reference images are recorded from camera. Errors in recording will lead to errors in recognition process.

32. Conclusion • We have designed and implemented the prototype of ART system • We have improved the card recognition algorithm with increased accuracy and efficiency • We have developed a rulebase game engine which is generic and extensible for card games

33. Future Work • Color Calibration • Improve in card recognition algorithm • Game Enhancement

34. End of Presentation Thank you very much

35. Q & A Session