Here a list to what I'm doing:

**Irace Model tests**: Test of changes in the model for categorical parameters. A multiplier is applied after at the end of each iteration based on the survivors of each iteration, the survivors can be defined as the ones alive at the end of the iteration or the ones that survive a test performed during the iteration.

The experiments consist in 20 irace runs performed for each test. The candidates obtained are compared later using a testing set. Comparing graphs and statistical tests can be found here [acotsp, spear, moaco]

- Sum of survivors : Survivors are defined as all the candidates that survived any elimination during the execution.
- Final survivors: Survivors are defined as the candidates alive at the end of the iteration.

- Moaco elite cadidates: A testing over the training set using all the elite candidates (obtained in the 20 runs described previously) was performed, in order to find out the reason why the models were not improving the model. Testing results and graphs

**Tuners Comparison**:Different benchmarks using available tuners under different configurations. Comparing graphs can be found here [acotsp-ftest, acotsp-ttest, spear-ftest,spear-ttest, moaco].

- Acotsp:
Instances [training / testing].
- Irace ftest:
- Tuned Parameters.
- Tests results & irace configuration files: [1000 / 5000] evaluations.

- Irace ttest:
- Tuned Parameters.
- Tests results & irace configuration files: [1000 / 5000] evaluations.

- Paramils:
- Tuned Parameters.
- Scenario files.
- Tests results: [1000 / 5000] evaluations.

- Smac v 2.02: Three experiments regular smac, without logarithm for the quality, no logarithm + default parameters + features per instance.
- Tuned Parameters
- Tuned Parameters + default.
- Instances + features
- Scenario files.
- Tests results:

- GGA (to perform)

- Irace ftest:
- Spear:
Instances [training / testing]
- Irace ftest:
- Tuned Parameters.
- Tests results & irace configuration files: [10000 evaluations / 172800 seconds].

- Irace ttest:
- Tuned Parameters.
- Tests results & irace configuration files: [10000 evaluations / 172800 seconds].

- Paramils:
- Tuned Parameters.
- Scenario files.
- Tests results: [10000 evaluations / 172800 seconds].

- Smac v 2.04 (runing node 4)

- GGA (to perform)

- Irace ftest:
- Moaco:
- Irace ftest :
- Tuned Parameters.
- Tests results & irace configuration files: [2000 / 5000] evaluations.

- Irace ttest :
- Tuned Parameters.
- Tests results & irace configuration files: [2000 / 5000] evaluations.

- Irace ftest :

**SCIP Tuning**: Tuning of scip solving the toll problem (Alessia) using generated instances [training / testing]. Comparing graphs can be found here.

- Basic scip: Default configuration of scip.
Tests results.

- Irace: Tuning made using a budget of 10000 experiments for 206 parameters and instances of size 20.
Tuned Parameters (only categorical).
Tests results & irace configuration file.

- Paramils
Tuned Parameters.
Scenario file.
Tests results.

**SMAC + CPLEX**: Tuning of smac using cplex testing with the amount of instances.

- Instances CAT 100: No apreciable difference in the tuning with all instances or only one.
- Instances CAT 200: Testing stopped to perform smac other tests in node 4.

**Irace parameter Analisys**: >Analisys some irace parameters (To be started).

- First elimination test: Irace using second instance for starting the first test.

- acotsp:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 5000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- spear:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 10000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- moaco

- acotsp:
- Iterations: Iterations were setted as 2 + 2*log2(nparameters).

- acotsp:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 5000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- spear:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 10000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- moaco

- acotsp:
- Min survivors: The number of survivors to finish an iteration was setted as 1.

- acotsp:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 5000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- spear:
- Tuned Parameters.
- Instances [training / testing].
- Tests results & irace configuration files: 5000 evaluations [ftest / ttest].
- Graphs:
- Individual survivors graphs [first-ftest / first-ttest / irace-ftest / irace-ttest].
- Combined survivor graphs [ftest / ttest].
- First elimination averages ftest [graph / data ], ttest [graph / data ].
- Testing results graph

- moaco

- acotsp:

**Irace numerical parameters model**: Modification of the irace model for numerical parameters! (reading CMAES).