Currently I am learning about Parameter Tuning. What I am exactly doing? here
What is Parameter Tuning?
Definition to be inserted....
Short Overview
There are several approaches proposed in the literature for tuning algorithms. Some of them are targeted to specific types of algorithms, specific number of parameters or particular kinds of parameters. Which method to use will depend on what are we trying to achieve in the tuning task and what are the resources available for it. Two big categories can be the starting point for classification:
- Model based tuning: Methods that construct incrementally*? a model of the performance of the parameter search space based on the information they get in the tuning process.
- Non model based tuning:
Different kinds of parameters can be distinguish according to their possible values:
- Numerical: parameters having an ordered range of possible values, integer or real, having a discrete or continuous range. Examples of this kind could be the temperature of a simulated annealing or the mutation rate of a evolutionary algorithm.
- Categorical: parameters having a set of possible values without any order. Examples can be
- Ordered: parameters having a set of ordered values (Nor sure if they exists as a category)
This parameters could be used under certain conditions (conditional parameters) or independly. A parameter is conditional when his use or possible set of values is influenced by other parameters of the algorithm to tune. Certain parameters are used only when certain component of the algorithm are activated, for example we could use the parameter "tabu list lenght" only if we activate the use of the "tabu list" in an algorithm. In other cases a parameter set of values can be restricted to other parameter asigned value, for example if the crossover rate is in a certain range of values we would like the mutation rate to be in a specific range of values.
Tools Available
Currently is possible to find a number tools free and publicly available for tuning algorithms. In this section I will incrementally add information about the tools available giving a short description of their approach and providing examples of their usage with external algorithms. In a first stage all the algorithms (if possible) will be tested in using the ACOTSP algorithm by Thomas Stutzle, which include different kinds of parameters.
- Relevance estimation and value calibration (REVAC)
- Bonesa
- ParamILS
- Sequential parameter optimization (SPO)
- Sequential Model-based Algorithm Configuration (SMAC)
- Iterated F-race (IRACE)
- Gender Based Genetic Algorithm (GGA)