This page is based on research in Real-Time Interactive Neuro-Evolution by Adrian Agogino and Kenneth Stanley
Here is a link to a program for viewing .avi movies on unix: XAnim Home Page
NEW: Press here to try an interactive Java demo with Peons
What is this?
The following demos depict real-time neuroevolution in action.
They are the result of research in the area of real-time
interactive neuro-evolution, which basically means evolving
brains in a live-action scenario, where a user can interact with them.
The movie clips are actual captures of the evolution occuring,
offering an unusually clear glimpse at the process.
If you find this interesting, take a look at
Note that the actual animation of the peons is smoother
in the program than in the movie clips. This allows them to
be more compact.
What is going on in the movies?
The demos actually depict a simple game.
There are a bunch of small dots, called peons.
(Note that occasioanlly we see a larger peon, shaped as a square)
There is a larger, red dot, called "the enemy."
There are one or two large yellow circles, called "goldmines."
There is a square in the middle, called "the base."
The peons always leave from the base. If they don't travel to
one of the goldmines in an alotted amount of time, they die.
Also, if the enemy touches a peon, it dies. When a successful
peon makes it to a mine, it is transported back to the base to try
again. Thus, a good peon can dodge the enemy and find a mine in a
Each peon has a neural network brain, which is a simple feed-forward
network. The input to the network is visual information about where
the enemy is, and where the mines are. The output is a movement decision.
However, keep in mind that the brains are initially completely random
in their connections, so peons initially do not know anything about
their environment, or how to interpret their sensors.
As the game progresses, peons are given a fitness rating. In the
movies below, the highest ranking peon at any given time is
represented as a large box, so that they stand out. High ranking peons
are allowed to reproduce to replace lower ranking peons that have
died. When peons reproduce, mutations occur. Hence, we
Here are the peon movies:
The population is composed entirely of peons with randomly connected
brains, so we don't expect to see anything intelligent. Not surprisingly,
we see a lot of peons scattering in random directions. This is how an
initial population looks before any adaptation occurs.
In only a matter of seconds, the peon population has learned to seek
out the nearest mine. However, this is a naive strategy- the enemy
never moved while they found this technique, and therefore they
are not prepared to react appropriately when the enemy finally attacks.
This illustrates the tendency of evolution to produce populations
that are deceptively simplistic, yet still successful.
Suddenly, the enemy attacks the naive population from the above clip.
The population has not learned to deal with a moving enemy, and therefore
it is quickly devastated. You can actually see the population dying out
here as its favorite strategy is completely outwitted.
In only 20 or 30 seconds, the peons devise a new strategy which beats the enemy's strategy. (Note that the enemy's strategy is fixed in this example.)
This illustrates how flexible online adaptation is, and how it is possible in
real-time. The strategy employed appears to confuse the enemy, rendering it
Peons Employ New Strategy
Note: The following two demos are with a new peon population starting
Orbital Training an Untrained Set
Once again we are observing an untrained population. However, notice
that the mine and the enemy are both "orbiting" the peons' home base.
This is a training scenario we developed to obtain a population that
could not rely on the destination being in any one location, yet still
had to avoid the enemy. This is a complicated prospect for the untrained
peons. Notice near the end of the movie where one aimless peon accidentally
wanders into the passing mine. This peon will now be honored with higher
selectivity in the future. Interestingly, its strategy is most likely
almost useless, since it just knows to wander in a certain direction,
which won't work with an orbiting mine. The situation will tend to select
for peons that are a little sensitive to the location to of the mine,
and eventually strengthen that sensitivity into a following instinct.
The peons in this clip have developed a very sophisticated and reactive
neural network to deal with a very quickly changing situation.
Please compare this clip to the previous one to see how far they have
come. Attaining this level of intelligence took a few minutes of training
with exactly the right parameters for fitness selection. (If the parameters
are changed, the peons are not able to attain this level.) Notice the
elegant balance between enemy avoidance and mine following that the
peons display. Please note that the mine stops midway through the clip.
This is part of the confusion the peons deal with- the mine is set to
stop moving at random times and then start up again later.
Peons after Orbital Training
Contact authors here: