How To Debug an Artificial Life Experiment

Posted October 13, 2015 by Charles Ofria in Information / 0 Comments

I often argue that a big advantage of working with computational artificial life systems is that you can perform evolution experiments that would be difficult (or impossible!) with natural organisms.  However, any time you want to use a digital system to understand how or why a particular trait or behavior evolves, your first challenge is to get that trait evolve at all!  Warning: this can be a lot harder than it sounds.  Of course, in this quest you will also learn a lot more about your study system, gain more insight into the dynamics you are investigating, and probably come up with at least half-a-dozen follow-up studies.

Below are the steps that I suggest for new projects.

Step 0: Set up your study and do some runs.  Formulate your hypothesis, pick the artificial life  system that you plan to use, and configure your digital environments (including controls) in the logical and straight-forward manner that you think is most likely to work.  Next, do the runs and when you see your target trait evolve,  jump to step 7 to continue your analysis!  (Just kidding…  the experiments never work out so cleanly on your first try.)

In the likely event your first runs didn’t work as expected, start your debugging by…

Step 1: Examine the organisms that DID evolve.  What tricks do they use to survive?  As far as you understand the environment that you configured, should these evolved organisms be doing as well as they are?  Make sure to actually do the math to calculate their real fitness.  For example, for one project we wanted to evolve organisms that would sense their environment and then perform the task that was most advantageous.  The selective pressures seemed clear: in each of two opposing environments, performing the correct task resulted in a 16-fold increase in replication rate, while performing the incorrect task meant instant death.  Each organism was equally likely to be born into either environment, so clearly they should sense and then always perform the “correct” task.  Yet, in each run, they simply picked a task and performed it unconditionally.  Once we worked out the math, we realized that 16 times as many offspring and only half of them dead was still an 8-fold increase in fitness.  This difference allowed us to rethink our environment and pick one that actually required phenotypic plasticity to be fit.

If looking at the evolved organisms doesn’t point out the problem…

Step 2: Try hand-writing an organism that displays the target behavior.  The goal of this strategy is to give you insight into why evolution is having a tough time producing your desired result.  In some cases, you may realize that the behavior you were looking for is nearly impossible in your system and will need to modify your setup.  Other times, you might realize that too many pieces would all need to come into place at once, and as such the behavior’s evolution won’t be possible unless other selective pressures exist to put building blocks in place.  For example, while the default instruction set in Avida theoretically allows organisms to perform multiplication, an initial attempt at writing such code by hand required over 60 lines of precisely ordered code — clearly far too many for natural selection to produce without additional goalposts along the way.

If writing your own organism doesn’t provide sufficient insight…

Step 3: Do a set of evolutionary runs starting with your hand-written organism as the ancestor.  If the target trait that you implemented evolves away, then either your environment isn’t selecting for what you believed it was or the original hypothesis that you were trying to test is false.  Either way, you need to re-think your experimental setup accordingly.  My favorite test here is to look at the line of descent for an evolved organism that lost the trait.  If the trait is binary, you should be able to find a parent organism that exhibits the trait and an offspring without it; then decipher why the latter individual was more successful.

If evolving the hand-written organism doesn’t identify the problem…

Step 4: Design a  configuration with all of the selective pressures in your favor.  For example, if you are trying to evolve an organism that altruistically gives up some of its own energy for the good of its neighbors, you might want to make the cost to the altruist be near zero and the benefit to the neighbors be substantial.  You can always reign in these numbers at a later time, but for now you are trying to understand how to get such behaviors to evolve at all.  If you still don’t see the behaviors appear under these extreme conditions, perhaps the behavior is not actually beneficial -or- it has too many pieces that all need to simultaneously come into place.

To test if the problem is that too many pieces need to evolve at once…

Step 5: Build a new mechanism that allows the organism to trivially perform the target function.  Build an atomic version of the trait that can be introduced to the genome with a single mutation.  For example, if you are evolving genetic programs (like in Avida), create an instruction that performs the full trait in a single, cheap step.  This technique will make it trivial to evolve the function (a single mutation) and as inexpensive as possible, thus removing the last of the simple barriers to it appearing.  You should also create a control instruction that is identical in most ways (for example, how it interacts with code around it) and has the same costs, but does NOT perform the target function.  If your new instruction fails to outperform the negative control, then the functionality is almost certainly not beneficial.

If the atomic version of this trait does get adopted…

Step 6: Perform tests to determine why the atomic version of the trait gets adopted.  Repeat the experiment while slowly raising the cost of the atomic version of the trait.  If the trait is only evolved in at low costs, then the underlying problem is that the trait was not as valuable as you originally believed.  Next, repeat the experiment again while decreasing the probability of the atomic version of the trait evolving in.  If it is only maintained at high probabilities of appearing, it is likely only beneficial in limited circumstances and thus isn’t always immediately selected.  It may also be possible to break down this atomic version into two or three components to determine if the underlying problem actually lies in only one aspect of the trait.

Now that your experiments are finally working…

Step 7: Finish the project. Do a detailed statistical analysis on the results (coming up with many side projects as you do… go to step 1 for each of them), write everything up, ???, publish!

Each of these steps taps into the key advantages of using a digital system by performing manipulations that are typically impossible in wet-lab experiments.  While the results of the debugging steps won’t typically be usable in the final paper, they will give you a much deeper insight into the dynamics of your system, likely prompt you into performing additional experiments, and make for a more informed and useful paper in the end.

Of course, some dynamics may be too complex for these techniques to help with, for example if you’re looking for a set of ecological interactions to arise.  Even if you correctly hand-code all of the organisms, the timing of the spatial positioning might still be off, and those are much harder to diagnose, often requiring more of a trial-and-error approach (or a systematic process of adding new dynamics and observing their effects on the behavior of your system!).  However, spatial structure, temporal fluctuations, and population composition are all aspects to consider when trying to diagnose your issues.

Do have any other tricks you use in your Artificial Life experiments that you find particularly valuable?  Let us know!

Charles Ofria

Charles Ofria is a professor of Computer Science at Michigan State University, director of the Devolab, and deputy director of the BEACON Center for the Study of Evolution in Action.

More Posts - Website

Leave a Reply