by Leslie Pérez Cáceres, Manuel López Ibáñez, Holger Hoos, Thomas Stützle
2016
| Table of Contents |
The irace package is a widely used automatic configuration method implementing various iterated racing procedures. The original irace was designed for the optimisation of the solution quality reached within a given running time, a situation frequently arising when configuring algorithms such as stochastic local search procedures. However, when applied to configuration scenarios that involve minimising the running time of a given target algorithm, irace falls short of reaching the performance of other general-purpose configuration approaches, since it tends to spend too much time evaluating poor configurations. In this article, we improve the efficacy of irace in running time minimisation by integrating an adaptive capping mechanism into irace, inspired by the one available in ParamILS, a rather different algorithm configuration method. We demonstrate that the resulting irace cap reaches a performance that is competitive with that of state-of-the-art algorithm configurators that have been designed to perform well on running time minimisation scenarios. We also investigate the behaviour of irace-cap in detail and contrast different ways of integrating adaptive capping into irace.
Keywords:automatic algorithm configuration, parameter tuning, runtime optimisation, irace
The details of the results of the experiments described can be found in the appendix.
We use the version 2.3.8 of Paramils and the version 2.10.03 of Smac.
| Regions 100 | Regions 200 | Corlat | Lingeling | Spear | |
|---|---|---|---|---|---|
| Target algorithm | CPLEX 12.4 | CPLEX 12.4 | CPLEX 12.4 | Lingeling (azf-9203284-140522) | Spear |
| Parameters | [irace | Smac, Paramils] | [irace | Smac, Paramils] | [irace | Smac, Paramils] | [irace | Smac, Paramils ] | [irace | Smac, Paramils] |
| Initial candidates | [irace] | [irace] | [irace] | [irace] | [irace] |
| Target runner / execution | [scripts* | target-runner ] | [scripts* | target-runner ] | [scripts* | target-runner ] | [scripts* | target-runner ] | [scripts* | target-runner ] |
| Cut off time | 5 sec. | 300 sec. | 300 sec. | 300 sec. | 300 sec. |
| Instances | instances.tar.gz | instances.tar.gz | instances.tar.gz | instances.tar.gz | instances.tar.gz |
| Scenario | [irace | Smac | Paramils]** | [irace | Smac | Paramils]** | [irace | Smac | Paramils]** | [irace | Smac | Paramils]** | [irace | Smac | Paramils]** |
* The scripts use the runsolver software version 3.3.4. Runsolver can be obtained in https://www.cril.univ-artois.fr/~roussel/runsolver/
** Execution line example for the experiments performed Smac and Paramils
The version 3.0 of the irace package including the capping procedure is currently under development. If you wish to test it, please write an email to: leslie.perez.caceres@ulb.ac.be. Information about the irace package is available in: http://iridia.ulb.ac.be/irace/. For questions about the package, please write in our google group: The irace package google group.
[1] López−Ibánez, M., Pérez Cáceres, L., Dubois-Lacoste, J., Birattari, M. and Stützle, T.: The irace Package: Iterated Racing for Automatic Algorithm Configuration, Operations Research Perspectives, 2016.
[2] Pérez Cáceres, L., López−Ibánez, M., Hoos, H., and Stützle, T.: The irace Package: An experimental study of adaptive capping in irace. Technical Report TR/IRIDIA/2016-007, IRIDIA, Université Libre de Bruxelles, Belgium, (2017).
[2] López−Ibánez, M., Pérez Cáceres, L., Dubois-Lacoste, J., Birattari, M. and Stützle, T.: The irace Package: User Guide. Technical Report TR/IRIDIA/2016-004, IRIDIA, Université Libre de Bruxelles, Belgium, (2016).
[3] Birattari, M., Yuan, Z., Balaprakash, P. and Stützle, T .: F-race and iterated F-race: An overview.. Experimental Methods for the Analysis of Optimization Algorithms, 311−336 (2010)
[4] Pérez Cáceres, L., López−Ibánez, M. and Stützle, T.: An analysis of parameters of irace In Proceedings of EvoCOP 2014 − 14th European Conference on Evolutionary Computation in Combinatorial Optimization, volume 8600 of LNCS, 37−48 (2014)