Compiling and running BBOB'2013 benchmark

Building and running the benchmark

The BBOB'2013 benchmark measures the efficiency of the optimization procedures on a series of functions in many dimensions. libcmaes has support for the benchmark and ranks among the best algorithms.

There's an install script in the repository. Do:

cd tests
./bbobsetup.sh

you can now benchmark any of the implemented flavors of CMA-ES (beware, this make take a while, ~hours):

./bbobexperiment -alg bipop

for the command above, results will be in repository bipop_bbob

See (1) for more information and details.

Results

As of July 2014, results on the noise-free version of the benchmark:

• all algorithms compared: http://juban.free.fr/stuff/libcmaes_bbob/bbob_comp.pdf
• active BIPOP-CMA-ES: http://juban.free.fr/stuff/libcmaes_bbob/abipop.pdf
• raw data: http://juban.free.fr/stuff/libcmaes_bbob/all_bbob.tar.gz

As of August 2014, results on the noisy version of the benchmark:

• all algorithms compared: http://juban.free.fr/stuff/libcmaes_bbob/bbob_noisy_comp.pdf
• active BIPOP-CMA-ES: http://juban.free.fr/stuff/libcmaes_bbob/abipop_noisy.pdf
• BIPOP-CMA-ES: http://juban.free.fr/stuff/libcmaes_bbob/bipop_noisy.pdf
• raw data: http://juban.free.fr/stuff/libcmaes_bbob/all_bbob_noisy.tar.gz

References

• (1) http://coco.gforge.inria.fr/doku.php?id=bbob-2013