CS510 AI and Games

CS510 AI and Games Final Report on Dec. 09 2008 Juncao Li

Agenda • About Advanced Protection • My Design and Work • Results and Evaluations • Conclusion and Discussion Computer Science, Portland State University

Advanced Protection (AP) • Author: Soren Johnson • Turn-based strategy game • Player makes strategy each turn • AI plays against player’s strategy • Static AI V.S. adaptive AI • Player learns the strategy of static AIs • Adaptive AI adapts based on users performance Computer Science, Portland State University

AP (Cont.) Computer Science, Portland State University

AP (Cont.) • Features • Each minion has a brain (automaton) encoded by a 128-bit string • Four behaviors depending on the input: • Move forward, turn right, turn left and do actions • 250 brains for a minion to choose • 20 hardcoded, 230 from genetic algorithm • During the play, brains are rated based on their performance against the player • Dynamically choose best-fit brains • Each player has their own easy and hard minion brains • Players does not loose or win too much Computer Science, Portland State University

Minion Inputs Computer Science, Portland State University

Why Brain Matters Computer Science, Portland State University

Why Brain Matters (Cont.) Computer Science, Portland State University

My Goal & Why • Design AIs that play as players • Try to learn the player’s strategy • Try to perform best on all 250 brains • Because Chaos selects brains based on player’s strategy • Win (defined later) brains as much as it could • Design benchmark for the AIs on each side • How well each AI performs • Learning about players helps game designs • Player will not like a game that they can never win! • Player will not like a game that they can easily win! Computer Science, Portland State University

Finite State Machine (FSM) • Code the game strategy in a FSM • Inputs or what matters • Treasury: Human and Chaos • Terrain and current human nodes on the map • Output • Where to place the units • What units to place • Based on current state • Allocate more farmers for earning money • Allocate more infantry for attacking Chaos Computer Science, Portland State University

FSM (Cont.) Computer Science, Portland State University

FSM (Cont.) • My FSM is static • Hard to make it dynamic and adaptive • Too many possibilities (24*24 size map plus human ……) • Works well as a benchmark • How well the Chaos AI performs • Train Neural Network (NN) • Hopefully, teach the NN to recognize the map Computer Science, Portland State University

Use Neural Network (NN) • Input nodes: MAP_SIZE+2 • Map size: 24*24 = 576 • Human+Chaos treasury: 1+1 • Output nodes: MAP_SIZE • Human units placement • One hidden layer: MAP_SIZE+2 • Normalize the inputs/outputs range: [0.0, 1.0] • Use other’s NN code: • AI Game Engine: well developed, but not efficient • Tim Jones book: simple and efficient NN Computer Science, Portland State University

Train the NN • Firstly, use the static FSM to train the NN • Random generated maps • Random initial Human/Chaos treasury • FSM generates outputs as the training case • Secondly, improve the NN • Randomly generate strategy • If it performs better than current NN • Train NN with the strategy Computer Science, Portland State University

Understand the NN Outputs • NN doesn’t always tell the exact answer • So we have to “guess” CNNPlayer::DoTurns(){ …… //normalize the output float low = actual[0]; float high = actual[0]; float avg = 0; for(i=0; i<OUTPUT_NEURONS; i++){ if(low>actual[i]) low=actual[i]; if(high<actual[i]) high=actual[i]; avg+=actual[i]/OUTPUT_NEURONS; } for(i=0; i<OUTPUT_NEURONS; i++){ if(actual[i]<=0) actual[i] = 0; else{ actual[i] = (actual[i]-low)/(high-low); //actual[i] = actual[i]/avg; } } …… } Human_TMatchHuman(float type){ float threashold = 0.2; type = type*HUMAN_TYPES; if(type<=threashold || type>HUMAN_TYPES+threashold) return NO_HUMAN; if(fabs(type-float(DRONE))<threashold) return DRONE; else if(fabs(type-float(MINE))<threashold) return MINE; else if(fabs(type-float(FARMER))<threashold) return FARMER; else if(fabs(type-float(INFANTRY))<threashold) return INFANTRY; else if(fabs(type-float(SETTLER))<threashold) return SETTLER; else if(fabs(type-float(ARMOR))<threashold) return ARMOR; else if(fabs(type-float(JAMMER))<threashold) return JAMMER; else if(fabs(type-float(ARTILLERY))<threashold) return ARTILLERY; return NO_HUMAN; } Computer Science, Portland State University

NN Outputs (4 hour & 1M inter.) Computer Science, Portland State University

Recall the FSM Computer Science, Portland State University

A Random NN Give You This! Computer Science, Portland State University

Evaluations • Test based on each turn • Only for each turn, the Chaos’ brain is certain • My AI performs well (win) if and only if: • Given random initial treasury and map • It has advantage against most Chaos’ brains (money earned) • Good performance on each turn leads to final win Computer Science, Portland State University

Statistics of My FSM We want this! Initial Treasury Not well dealing with money between this range Computer Science, Portland State University

Statistics of My NN AI We want this! Initial Treasury Computer Science, Portland State University

Conclusion • Studied AP • Adaptive turn-based strategy game • Static FSM • Benchmark • Train NN • NN • Developed a demo • Trained by FSM • Need more time to train it Computer Science, Portland State University

Discussion • Static FSM • Efficient to code • Pros: do exactly what you want • Cons: do exactly what you want • NN • Like dealing with a child • Patient • Be careful what you feed to it • Try to understand a child’s language • More trained, usually better performance Computer Science, Portland State University

Discussion (Cont.) • Coding & debugging issues • Error in coding NN is not easy to be detected • Wrong training data • Inconsistent between training inputs and using inputs • Be careful about important parts • Training data • Training process • Set breakpoint to check • Code review • Train it by the same example, see if it adapts as expected Computer Science, Portland State University

Questions ? Computer Science, Portland State University

CS510 AI and Games