A practical introduction to irace

IRIDIA

Introduction

The irace package is an R implementation of the iterated racing procedure for automatic algorithm configuration. This tutorial will help you install and run it on two well-known optimization software:


More information on irace can be found on its official webpage: http://iridia.ulb.ac.be/irace/

Installation

Warning The irace package requires the R language for statistical computation interpreter.

Installing R

There are official instructions available. We give below a quick installation guide that will work in most cases.

GNU/Linux

You should install R from your package manager. On a Debian/Ubuntu system it will be something like:

$ sudo apt-get install r-base

Once R is installed, you can launch R from the terminal and from the R prompt install the irace package. See instructions below.

OS X

You can install R directly from a CRAN mirror (the link is of the Belgian one):

http://cran.freestatistics.org/bin/macosx/

Alternatively, if you use homebrew, you can just brew the R formula from the science tap (unfortunately it does not come already bottled so you need to have Xcode installed to compile it):

$ brew tap homebrew/science
$ brew install r

Once R is installed, you can launch R from the Terminal (or from your Applications), and from the R prompt install the irace package. See instructions below.

Windows

You can install R from a CRAN mirror (the link is of the Belgian one):

http://cran.freestatistics.org/bin/windows/

Once R is installed, you can launch the R console and install the irace package from it. See instructions below.

Installing the irace package

There are two methods to install the irace R package on your computer:

  1. Install within R (automatic download): 
    $ R
> install.packages("irace")
    select a mirror close to you, and test the installation with 
    > library(irace)
> CTRL+d
  2. Manually download the package from CRAN and invoke at the command-line: 
    $ R CMD INSTALL <package>
    where <package> is one of the three versions available: .tar.gz (Unix/BSD/GNU/Linux), .tgz (MacOS X), or .zip (Windows).

If the package fails to install because of insufficient permissions, you need to force a local installation by doing:

$ mkdir ~/R
$ R CMD INSTALL --library=~/R irace.tar.gz
$ export R_LIBS=~/R:${R_LIBS}

Once installed, test that it is working by doing:

$ R
> library(irace)
> system.file(package="irace")
[1] "~/R/irace"
GNU/Linux and Mac OS X

The previous command tells you the installation directory of irace. The last step is to add this directory to the PATH environment variable so it can locate the irace wrapper.

Warning The wrapper invokes the R interpreter in command-line mode and launches irace on your current dir.

Save that path to a variable, and add it to your .bash_profile, .bashrc or .profile:

export IRACE_HOME=~/R/irace/ # Path given by system.file(package="irace")
export PATH=${IRACE_HOME}/bin/:$PATH
# export R_LIBS=~/R:${R_LIBS} # Only if local installation was forced

Now you need only to reload the file you just edited, or open a new terminal, and you should be able to incoke irace:

$ source ~/.bash_profile
$ irace --help
Windows

Unfortunately, the command-line wrapper does not work in Windows. To launch irace, you need to open the R console and execute:

R> library(irace)
R> irace.cmdline("--help")

Using irace to tune ACOTSP

Info To download the material required for this exercise, click here.

After uncompressing the acotsp-example.tar.gz file, go to the ACOTSP-1.04-tuning directory and compile ACOTSP using the make command:

     $ tar xzvf acotsp-example.tar.gz
     $ cd acotsp-example
     $ cd ACOTSP-1.04-tuning
     $ make

Go back to the directory where you uncompressed acotsp-example.tar.gz and run irace.

     $ cd ..
     $ irace --parallel 4

Info The --parallel option makes irace use multiple cores. In case you have a dual core processor, use --parallel 2.

Watch irace run and analyze the configurations selected at the end of each iteration. When irace finishes, compare the candidates selected and try to understand the logic behind those parameter settings.

Using irace to tune NSGA-II

Info To download the material required for this exercise, click here.

After uncompressing the nsga2-example.tar.gz file, go to the nsga2-gnuplot-v1.1.6 directory and compile NSGA-II using the make command:

     $ tar xzvf nsga2-example.tar.gz
     $ cd nsga2-example
     $ cd nsga2-gnuplot-v1.1.6
     $ make

Go back to the directory where you uncompressed nsga2-example.tar.gz. You will now need to compile the performance assessment tools for computing the hypervolume. First, compile the nondominated program, used to normalize and filter the approximation sets.

     $ cd ..
     $ cd mo-tools-1.0svn100-src
     $ make nondominated

Info The march=native option in the mo-tools Makefile asks the compiler to detect your architecture so it can generate optimized code. In case your compiler does not support this option (triggering an error), we recommend you manually specify your archicture, e.g., march=i386 or march=x86-64.

The second tool to be compiled is the hypervolume program:

     $ cd ..
     $ cd hv-2.0rc1-src
     $ make

Go back to the directory where you uncompressed nsga2-example.tar.gz and run irace.

     $ cd ..
     $ irace --parallel 4

Info The --parallel option makes irace use multiple cores. In case you have a dual core processor, use --parallel 2.

Watch irace run and analyze the configurations selected at the end of each iteration. When irace finishes, compare the candidates selected and try to understand the logic behind those parameter settings.

Tuning irace using a different hypervolume computation method

Now that you have tested irace using the normalized hypervolume, the next step is to test it without normalization. Edit the tune-conf, locate at the directory where you uncompressed nsga2-example.tar.gz, and edit the two lines below:

     #hookRun <- "./hook-run2"		# uncomment this line
     hookEvaluate <- "./hook-evaluate"  # comment this line

Run irace again and compare results.