A Study on the Design and Implementation of an Artificially Intelligent Control System

1. A Study on the Design and Implementation of an Artificially Intelligent Control System Sagar Indurkhya

2. Purpose To construct a virtual simulation of boids and to test a proposed model for the boids’ control system.

3. Background • Boid • An autonomous unit that can independently act with its environment. • Artificial Neural Networks • An Artificial Neural Network is graph of artificial neurons (vertices) and the • connections between them (edges). • Supervised Artificial Neural Networks • Used when training data is available. • Inputs are fed into the network, and the values that the network outputs • are compared to the reference outputs. • A back-propagation algorithm is run adjusting the weights of the network • such to minimize error. • Unsupervised Artificial Neural Networks • Used when training data is not available. • The network adapts purely in response to its inputs. • Used for self organizing maps, and employed in competitive learning.

4. Feed-Forward Architecture • Consist of simple layer by layer networks. • Output flows from the input layer, through the hidden layers, and finally to the output layer. Recurrent Network Architecture • Neurons may receive input not only from the neurons before them, but from any neuron within the network. • An example of a recurrent (biological) neural network is the human brain, in which billions of neurons for trillions of connections between each other. Genetic Algorithm • A method to generate approximate solutions to problems that require optimization and searching. • Model the natural process of evolution by “crossing breeding” the most successful solutions to form a new generation of solutions.

5. Feed Forward Neural Network Recurrent Neural Network Red = Input Layer Light Green = Hidden LayersLight Blue = Output Layer

6. Prediction Network • A supervised recurrent neural network that is essentially a vehicle for time series • analysis. • Takes in several environment states as input, and outputs one action set. • It then predicts the next environment state, supposing the bot would • perform the action set with the most recent environment state. • The extra environment states represent immediate memory, with long term • memory stored in the inter-neuron connections.

7. Situation Analysis Network • This feed forward neural network takes in an environment state as input, and provides a numerical output, a floating point value between 0.0 and 1.0, allowing the control system to compare and judge between multiple environment states. • This network is not trained with the standard Error Back Propagation algorithm; rather the weights are strung together to form a chromosome, which allow a Genetic Algorithm to employ mutation operators upon it to generate many derivative chromosomes for future generations.

8. Proposed Model for the Control System The model for a Control System revolves around the method to play a game of chess. Chess, a perfect game, defines all possible moves for both players. In this simulation, as in real life, we often know what we can do, but we do not fully understand how the environ- ment acts. This model calls on the control system to predict how the environment will react. The Prediction network takes in a series of environment states and an action set, and predicts what the next environment state will be. By repeating this process, we can predict a cause-effect sequence of events. By trying different action sets at each point, we can generate a game tree, as in chess. The Situation Analysis network is used to evaluate a given environment state as extremely desirable (1.0) to least desirable(0.0). The difficulty in this model is that both networks must start completely randomized, and the boids must essentially learn on the go.

9. Method • The simulation constructed generations of bots, each generation having 200 bots. • These bots interacted with each other, and lived for at most 1,000 iterations. • Each iteration, the bot took in an input vector (represented internally as an Environment State), processed the input vector through its control system, and declared an Action Set containing its intentions to its host shell (the body framework) as per how to move, rotate, etc. • Finally, the simulation executed each bot’s physical actions and the next iteration began. • At the end of each generation, every bot’s chromosomes were entered into a common gene pool, from which offspring were be generated.

11. Experiment • Conduct a single trial of four populations, A, B, C, and D where each set is run in an isolated environment for 1,000 generation, each consisting of 10,000 iterations. • Merge Population A with Population B, and Population C with Population D, allowing the two merged populations to run for 1,000 generations, each consisting of 10,000 iterations. • Split the two merged populations back into 4 populations, A’, B’, C’, and D’. 4. Go back to Step 1, and continue the process for as long as possible while computational resources were available. 5. Every 100 generations, the simulation generates a log file, which is transferred to another computer, from which an examiner can observe and note any patterns or signs of intelligence that the bots displayed. The log file also keeps a record of the weights in a bot’s prediction network, so that the examiner can note any connections that were very close to zero. This assisted in helping us map out the development of inter-neuron connections, and let us search for any patterns that may have emerged.

13. Objectives for Intelligence • A large clustering of bots in a small area, not firing towards each other, but possibly firing against outside bots would be displaying cooperative group work. This would be analogous to a group of people gathering together to form a community, in which they all work together for the common good. • Bots attacking other bots from behind might suggest a display of spatial awareness, as well as an acknowledgement of other bots’ motives. • A decrease in the health of a bot might be an indicator of an interesting encounter between multiple bots, and may lead to displays of intelligence. Every time a bots health suffers, the post-computation analysis guidance program should focus in on the immediate situation which foreshadowed the decrease in the bot’s health. Thus the examiner can assess the circumstances that led to the decrease in health, dismiss any random shots, and note situations that may display cooperative group work or forward thinking. • At the end of each generation, the bots with the most health are tracked through their life span, to see if they actually exhibited any interesting logic, or simply avoided others. • As the bots can communicate with each other, the sound channels are searched for any patterns.

14. Display Interface This program is used to view the output of the actual simulation, as the simulation cannot run in real time (it isn’t possible to view the results live). It enables one to closely examine the data, looking back and forth through the recreated movie.

15. Results In executing this experimental simulation, we found that the runtime was much longer than originally predicted. After running a 200 bot-simulation, we found that one generation took more than an hour to run. After running a small number of generations, we decided that the experiment needed to be revised to accommodate the apparently enormous runtime the simulation would require. We are in the process of doing so (see Discussion of Results). After looking at what log files had been produced, we found that as we expected, the bots at first simply randomly moved, and operated their firing mechanism randomly. Although we were only able to run a few generations due to limits on computational resources, it seemed as though perhaps the bots were becoming aware of their motor skills. This suggests that it may be beneficial to give bots independent time to explore and refine motor skills. Furthermore, no advantage was found in automatically zeroing weights in the neural network that headed towards zero. The results as of yet indicate that perhaps the proposed model should be implemented directly (building the boids physically so that the laborious simulation computations can be discarded) would provide better feedback. Furthermore, the objectives for intelligence should be revised.

16. Statement of Conclusion Although our results thus far have not shown any evidence of intelligence emerging, they strongly support our prior expectations for the behavior of the boids: the boids first moved randomly, and later began to demonstrate control of motor skills. Limitations in computing resources were the biggest hindrance to the simulation. Both neural networks and genetic algorithms tend to be slower than many other approaches, although they appear to be more practical for applications on a scale such as this, where handcrafted algorithms would be far to complex to be a viable option. The project can be improved by building physical implementations of the boids, so that the computations required for the simulation need not take place. Furthermore, it would be possible to determine other real world engineering concerns we should take into consideration. Finally, as the proposed model for a control system is meant to be applied generically to any situation, it would be wise to write various simultations for testing. It is hoped that the model for a control system can be applied to developing such as nanobots or autonomous cars and planes. In these situations, having a central hub to carry out computations and make decisions is not possible. It is far easier to let each individual boid make their own decisions in favor of themselves. For example, nanobots would be too small to contain sophisticated radio communications equipment to communicate with one another. However, if they act independently, they can accomplish difficult tasks via a brute force approach.