Benchmarking Derivative-Free Optimization Algorithms

Jorge J. Moré and Stefan M. Wild

This page contains supplemental information for the paper:

• Benchmarking Derivative-Free Optimization Algorithms, Mathematics and Computer Science Division, Argonne National Laboratory, Preprint ANL/MCS-P1471-1207, December 2007. [PDF]

Users interested in the use of the techniques described in this paper (data and performance profiles) for benchmarking derivative-free solvers should consult the results of our (ever-changing) shootout of derivative-free solvers. We hope to update these results as additional codes are developed and submitted.

The following information is provided to encourage the benchmarking of derivative-free optimization solvers. Comments, questions, and suggestions should be addressed to Stefan Wild at

Benchmark Problems

The following source files were used to define the benchmark problems:
• dfovec [Fortran]
  Code producing component vectors for the 22 CUTEr problems considered.
• dfoxs [Fortran]
  Code specifying the standard starting points.
• dfo.dat [Data file]
  Data file specifying the benchmark problem set P through the integer parameters (nprob, n, m, ns).

Additional details concerning the problems may be found on the CUTEr webpage.

Solvers

The following solvers were used to obtain the performance data:
• nmsmax [Matlab]
  Source code for this Nelder-Mead algorithm can be obtained via the Computation Toolbox of N Highman.
• appspack_serial [C++]
  Source code and documentation for the APPSPACK 5.0.1 solver can be obtained via the APPSPACK web page.
• newuoa [Fortran]
  Source code for this trust-region algorithm can be obtained directly from Mike Powell.

Data Profiles

• Smooth optimization problems
• Non-stochastic noisy optimization problems
• Non-differentiable optimization problems
• Stochastic noisy optimization problems

Performance Profiles

• Smooth optimization problems
• Non-stochastic noisy optimization problems
• Non-differentiable optimization problems
• Stochastic noisy optimization problems