This research is supported in part by the National Science Foundation under grant IIS-0083776 (and previously under IRI-9504317) and the Texas Higher Education Coordinating Board under grant ARP-003658-476-2001. Some of our projects are described below; for more details and for other projects, see publications in Neuroevolution Methods and Applications.
Automatic process control in the metallurgical and chemical industries is difficult for two reasons: (1) the processes are nonlinear, and (2) they change over time. In this project, neuroevolution techniques were developed to deal with both problems, using a bioreactor process as a domain. Symbiotic evolution of neural networks finds a nonlinear controller efficiently, and particle swarm optimization allows it to adapt to changes in the process. The resulting system learns to control the highly nonlinear system off-line, and adapts on-line to process changes while simultaneously avoiding hard operating constraints.
To obtain best performance in a multi-processor chip, on-chip resources such as cache memory and off-chip bandwidth must be managed dynamically. In this project, ESP is used to evolve neural networks to decide how to continuously reallocate the available cache banks to processors. Trace-based simulations show that significant performance increases over static allocation are possible this way.
Syllable systems across languages share a number of common patterns. A particularly compelling explanation for these patterns is that they orginate from constraints provided by the perceptual and articulatory systems of language users. In this research, we use genetic algorithms to examine how a few experimentally defined perceptual and articulatory constraints on syllables interact to produce different relative distributions of syllable types in evolved vocabularies. The goal is to show how both language regularity and variation arise from optimizing the sound system under these constraints.
In many real world optimization problems, such as resource management and manufacturing, the optimization has to be done under uncertainty. Uncertainty makes the task nonlinear, and standard methods such as Linear Programming do not perform very well. In this project, the aim is to evolve neural networks with ESP to optimize a manufacturing process (of aluminum recycling). The goal is to learn to compensate for the uncertainty, and utilize hidden regularities in the data. With a small number of variables and with accuracy limited to a small number of values, ESP outperforms Linear Programming, but its performance is not significantly better with several highly accurate variables. This result is both a promise and a challenge for future neuroevolution research.
Music composition is a domain well-suited for evolutionary reinforcement learning. It is possible to write down rules for good melodies, and those rules can be used as a fitness function. Our initial results shows that the networks learn to express primitive rules on tonality and rhythm this way. Furthermore, by including rules for the style of a particular composer (e.g. Bela Bartok) the output melodies locally resemble the composer's work. Several interesting structures are discovered: for instance, the concept of transposition is naturally adopted by the model to reproduce familiar patterns with slight variations.
Using the SANE and ESP methods, we have evolved networks to play go on small boards. When evolved against a computer program, the networks quickly learn to utilize weaknesses in the opponent, and exhibit aspects of general go-playing behavior. With fitness sharing, shared sampling, and hall-of-fame techniques, networks can be evolved against each other, and they reach a higher playing level that when evolved against a single opponent. Evolving specialists for different parts of the board also leads to improved performance.
Natural deduction is essentially a sequential decision task, similar to many game-playing tasks. In this project, SANE was used to evolve neural networks to identify steps in a natural deduction proof. Incremental evolution through progressively more challenging problems resulted in more effective networks than does direct evolution, and an effective network was evolved faster when the network was allowed to ``brainstorm'' or suggest any move regardless of its applicability, even though the highest-ranked valid move is always applied. This way evolution resulted in neural networks with human-like reasoning behavior.
In standard neuro-evolution, the objective is to evolve a network that best handles a given task. Although this approach is useful for static tasks, it does not work well in real-time domains where the environment (and therefore the task) can vary. Furthermore, if the real-time domain is interactive, the task is unpredictable because the user can change his/her behavior at will. We have tackled this problem by introducing a method for real-time interactive neuro-evolution, and testing the method through a real-time interactive gaming scenario. As the environment changes, the population evolves along with it and can cope with the task. We show that this method is superior to standard neuro-evolution techniques in the paper below.
A version of SANE was used to evolve feedback controllers for chaotic systems. Standard methods for this task require an analytical description of the system near the fixed point. In contrast, the SANE networks were able to stabilize unstable fixed points for logistic and Henon maps under noise, and without prior knowledge of the system such as the location of the points or the dimensionality of the system. Such robustness suggests that the method could scale up well to real-world applications.
SANE has been used to evolve robot controllers in two simulated environment: reaching for a target while avoiding obstacles with the Simderella robot arm, and for running in a maze environment with the Khepera miniature mobile robot simulator. In both cases, evolution discovered a general, robust strategy that would be difficult to learn in any other way or even design by hand.
When evolved against a positional strategy of Iago, evolution discovered how to turn the initial disadvantage in material into a novel advantage: mobility strategy. Neuroevolution can also be used to improve play of existing programs in novel ways: for example, a network can be evolved to filter the information minimax search is allowed to see, and thereby compensate for deficiencies in minimax and the heuristic function.