The irace Package: Iterated Racing for Automatic Algorithm Configuration: Supplementary material

by Manuel López Ibáñez, Leslie Pérez Cáceres, Jérémie Dubois-Lacoste, Mauro Birattari, Thomas Stützle

Table of Contents
  1. Abstract
  2. Scenarios
  3. Package
  4. References

1. Abstract:

Modern optimization algorithms typically require the setting of a large number of parameters to optimize their performance. The immediate goal of automatic algorithm configuration is to find, automatically, the best parameter settings of an optimizer. Ultimately, automatic algorithm configuration has the potential to lead to new design paradigms of optimization software. The irace package is a software package that implements a number of automatic configuration procedures. In particular, it offers iterated racing procedures, which have been used successfully to automatically configure various state-of-the-art algorithms. The iterated racing procedures implemented in irace include the iterated F-race algorithm and several extensions and improvements over the original iterated F-race proposal. In this paper, we describe the rationale underlying the iterated racing procedures and introduce a number of recent extensions. Among these, we introduce a restart mechanism to avoid premature convergence, the use of truncated sampling distributions to handle correctly parameter bounds, and an elitist racing procedure for ensuring that the best configurations found are also those evaluated in the higher number of training instances. We experimentally evaluate the most recent version of irace and demonstrate with a number of example applications the use and potential of irace, in particular, and automatic algorithm configuration, in general.

Keywords:automatic algorithm configuration, racing, parameter tuning

2. Scenarios:

Target algorithm ACOTSP website executable MOACO website
Parameters acotsp-parameters.txt spear-parameters-cat.txt moaco-parameters-full.txt
Target runner target-runner target-runner target-runner
Cut off time 20 sec. 300 sec. variable (see target-runner)
Target evaluator - - target-evaluator
Instances instances.tar.gz | optimum instances.tar.gz instances.tar.gz
Scenario scenario.txt scenario.txt
Other - - Hypervolume

3. The irace package:

The version 2.0 of the irace package will be available soon in CRAN, a preliminary version can be downloaded here.

4. References:

[1] López−Ibánez, M., Pérez Cáceres, L., Dubois-Lacoste, J., Birattari, M. and Stützle, T.: The irace Package: User Guide. Technical Report TR/IRIDIA/2016-004, IRIDIA, Université Libre de Bruxelles, Belgium, (2016).
[2] Birattari, M., Yuan, Z., Balaprakash, P. and Stützle, T .: F-race and iterated F-race: An overview.. Experimental Methods for the Analysis of Optimization Algorithms, 311−336 (2010)
[3] Pérez Cáceres, L., López−Ibánez, M. and Stützle, T.: An analysis of parameters of irace In Proceedings of EvoCOP 2014 − 14th European Conference on Evolutionary Computation in Combinatorial Optimization, volume 8600 of LNCS, 37−48 (2014)