A Study on the Design and Implementation of an Artificially Intelligent Control System

1. A Study on the Design and Implementation of an Artificially Intelligent Control System Sagar Indurkhya The North Carolina School of Science and Mathematics

2. Purpose • To design a theoretical model for an artificially intelligent control system. • To construct a controlled virtual simulation • in which a population of boids can exist and interact with each other. • To test the control system on the boids and study both the practicality and efficiency of the control system.

3. Background • Boid • An autonomous unit that can independently act with its environment. • Artificial Neural Networks • An Artificial Neural Network is graph of artificial neurons (vertices) and the • connections between them (edges). • Supervised Artificial Neural Networks • Used when training data is available. • Inputs are fed into the network, and the values that the network outputs • are compared to the reference outputs. • A back-propagation algorithm is run adjusting the weights of the network • such to minimize error.

4. Unsupervised Artificial Neural Networks • Used when training data is not available. • The network adapts purely in response to its inputs. • Used for self organizing maps, and employed in competitive learning. Feed-Forward Architecture • Consist of simple layer by layer networks. • Output flows from the input layer, through the hidden layers, and finally to the output layer. Recurrent Network Architecture • Neurons may receive input not only from the neurons before them, but from any neuron within the network. • An example of a recurrent (biological) neural network is the human brain, in which billions of neurons for trillions of connections between each other. Genetic Algorithm • A method to generate approximate solutions to problems that require optimization and searching. • Model the natural process of evolution by “crossing breeding” the most successful solutions to form a new generation of solutions.

5. Feed Forward Neural Network Recurrent Neural Network Red = Input Layer Light Green = Hidden LayersLight Blue = Output Layer

6. Prediction Network • A supervised recurrent neural network that is essentially a vehicle for time series • analysis. • Takes in several environment states as input, and outputs one action set. • It then predicts the next environment state, supposing the bot would • perform the action set with the most recent environment state. • The extra environment states represent immediate memory, with long term • memory stored in the inter-neuron connections.

7. Situation Analysis Network • This feed forward neural network takes in an environment state as input, and provides a numerical output, a floating point value between 0.0 and 1.0, allowing the control system to compare and judge between multiple environment states. • This network is not trained with the standard Error Back Propagation algorithm; rather the weights are strung together to form a chromosome, which allow a Genetic Algorithm to employ mutation operators upon it to generate many derivative chromosomes for future generations.

8. Proposed Model for the Control System • The model for a Control System revolves around the method to play a game of chess. • Chess, a perfect game, defines all possible moves for both players. • In this simulation, as in real life, we often know what we can do, but we do not fully understand how the environment acts. • This model calls on the control system to predict how the environment will react. • The Prediction network takes in a series of environment states and an action set, and predicts what the next environment state • will be. • By repeating this process, we can predict a cause-effect • sequence of events.

9. Continued - Proposed Model for the Control System • By trying different action sets at each point, it is possible to generate a game tree, as in chess. • The Situation Analysis network is used to evaluate a given environment state as extremely desirable (1.0) to least desirable(0.0), enabling us to cutoff the game tree when we • have reached a predefined look ahead depth. • The difficulty in this model is that both networks must start completely randomized, and the boids must essentially learn • on the go.

10. Method • The simulation constructed generations of bots, each generation having 200 bots. • These bots interacted with each other, and lived for at most 1,000 iterations. • Each iteration, the bot took in an input vector (represented internally as an Environment State), processed the input vector through its control system, and declared an Action Set containing its intentions to its host shell (the body framework) as per how to move, rotate, etc. • Finally, the simulation executed each bot’s physical actions and the next iteration began. • At the end of each generation, every bot’s chromosomes were entered into a common gene pool, from which offspring were be generated.

12. Experiment 1. Conduct a single trial of four populations, A, B, C, and D where each set is run in an isolated environment for 1,000 generation, each consisting of 10,000 iterations. 2. Merge Population A with Population B, and Population C with Population D, allowing the two merged populations to run for 1,000 generations, each consisting of 10,000 iterations. 3. Split the two merged populations back into 4 populations, A’, B’, C’, and D’. 4. Go back to Step 1, and continue the process for as long as possible while computational resources were available. 5. Every 100 generations, the simulation generates a log file, which is transferred to another computer, from which an examiner can observe and note any patterns or signs of intelligence that the bots displayed. • The log file also keeps a record of the weights in a bot’s prediction network, so that the examiner can note any connections that were very close to zero. • This assisted in helping us map out the development of inter-neuron connections, and let us search for any patterns that may have emerged.

14. Objectives for Intelligence • A large clustering of bots in a small area, not firing towards each other, but possibly firing against outside bots would be displaying cooperative group work. • Bots attacking other bots from - a display of spatial awareness, and acknowledgement of other bots’ motives. • A decrease in the health of a bot might be an indicator of an interesting encounter between multiple bots, and may lead to displays of intelligence. • At the end of each generation, the bots with the most health are tracked through their life span, to see if they actually exhibited any interesting logic, or simply avoided others. • As the bots can communicate with each other, the sound channels are searched for any patterns.

15. Display Interface This program is used to view the output of the actual simulation, as the simulation cannot run in real time (it isn’t possible to view the results live). It enables one to closely examine the data, looking back and forth through the recreated movie.

16. Results • In executing this experimental simulation, we found that the runtime was much longer than originally predicted. • After running a 200 bot-simulation, we found that one generation took more than an hour to run. After running a small number of generations, we decided that the experiment needed to be revised to accommodate the apparently enormous runtime the simulation would require. • After looking at what log files had been produced, we found that as we expected, the bots at first simply randomly moved, and operated their firing mechanism randomly. In time however, it was noted that some motor abilities may have been developed. • No advantage was found in automatically zeroing weights in the neural network that headed towards zero.

17. Discussion of Results • Although we were only able to run a few generations due • to limits on computational resources, it seemed as though • perhaps the bots were becoming aware of their motor skills. • This suggests that it may be beneficial to give bots independent time to explore and refine motor skills. • The results as of yet indicate that perhaps the proposed model should be implemented directly (building the boids physically so that the laborious simulation computations can be discarded) would provide better feedback. • Furthermore, the objectives for intelligence should be revised.

18. Statement of Conclusion Although the results obtained thus far have not shown any evidence of intelligence emerging, they strongly support our prior expectations for the behavior of the boids: the boids first moved randomly, and later began to demonstrate control of motor skills. Limitations in computing resources were the biggest hindrance to the simulation. Both neural networks and genetic algorithms tend to be slower than many other approaches, although they appear to be more practical for applications on a scale such as this, where handcrafted algorithms would be far to complex to be a viable option.

19. Continued - Statement of Conclusion The project can be improved by building physical implement- ations of the boids, so that the computations required for the simulation need not take place. Furthermore, it would be possible to determine other real world engineering concerns we should take into consideration. Finally, as the proposed model for a control system is meant to be applied generically to any situation, it would be wise to write various simultations for testing. It is hoped that the model for a control system can be applied to developing such as nanobots or autonomous cars and planes. In these situations, having a central hub to carry out computations and make decisions is not possible. It is far easier to let each individual boid make their own decisions in favor of themselves. For example, nanobots would be too small to contain sophisticated radio communications equipment to communicate with one another. However, if they act independently, they can accomplish difficult tasks via a brute force approach.

20. Applications The model is intended for systems in which the use of a central hub for coordination between a large number of boids is impractical. Possible applications include: • Nanobots – it is far easier to construct a nano-scopic boid that makes its own decisions than it is to engineer the communications equipment required to relay information back and forth to a central hub. • UCAV – One of the major concerns for armed UCAV’s (Unmanned Combat Air Vehicles) is the ability to make a critical decision as per firing upon targets without the close supervision of a human. This model would aid UCAV’s in taking more decisive actions, as well as letting them communicate with each other in a manner analogous to the method used by human soldiers. • Autonomous Automobiles – As future trends grows clear, autonomous automobiles are likely to come within the next two decades. This model would allow cars to make decisions with driving while requiring only knowledge about its immediate surroundings. With the number of automobiles on the road, this is likely the most practical approach to this goal.

21. Acknowledgements A special thank you to: • Dr. Morrison from the North Carolina School of Science and Mathematics • Dr. Subramanian of the University of North Carolina at Charlotte • Dr. Halpin of the North Carolina School of Science and Mathematics • Ms. Parijata Mackey from the University of Chicago • The Research Chemistry class of the North Carolina School of Science and Mathematics