by Leslie Pérez Cáceres, Manuel López Ibáñez,
Thomas Stützle
2013
Table of Contents |
The irace package implements a flexible tool for the automatic configuration of algorithms. However, irace itself has specific
parameters that enable the customization of the search process according to the tuning scenario. In this paper, we analyze five parameters of
irace: the number of iterations, the number of instances seen before the first elimination test, the maximum number of elite configurations, the
statistical test and the confidence level of the statistical test. These parameters define some key aspects of the way irace searches and identifies
good configurations. Originally, their values have been set based on rules of thumb and an intuitive understanding of the configuration process.
This work aims at giving insights about the sensitivity of irace to these parameters in order to provide guidance for their settings and possible
further improvements of irace.
Keywords: automatic algorithm configuration, parameter tuning, irace, racing algorithms
Configuration Scenario | Parameters | Training Set | Test Set | Objective | Cut Off Time | Configuration Budget |
---|---|---|---|---|---|---|
ACOTSP | 11 parameters | 50 TSP instances | 250 TSP instances | Solution Quality | 20 sec. | 5000 evaluations |
SPEAR | 26 parameters | 302 SAT instances | 302 SAT instances | Runtime | 300 sec. | 10000 evaluations |
MOACO | 16 parameters | 60 TSP instances | 60 TSP instances | Solution Quality (hypervolume) | 4*(instance_size/100)^2 sec. | 5000 evaluations |
ACOTSP is a software package that implements various ant colony optimization (ACO) algorithms for solving the Traveling Salesman Problem (TSP). We used the version 1.02, available in the following link.
SPEAR is a tree search solver for SAT problems. We used the version 2.7, available in the following link.
MOACO is a framework of multi-objective ACO algorithms. We used the version 0.1, available in the following link.