Add CMA-ES based suggestion service. #1100
Description
/kind feature
Describe the solution you'd like
The covariance matrix adaptation evolution strategy (CMA-ES) [1] is one of the most promising black-box optimization methods, and it has shown the best performance out of over 100 optimization methods for a variety of BBO problems [2].
In this issue, I propose to add CMA-ES based suggestion service. If approved, I'll send a pull request.
[1] N. Hansen. The CMA Evolution Strategy: A Tutorial. arXiv:1604.00772, 2016.
[2] Loshchilov, I., Schoenauer, M., and Sebag, M. Bi-population CMA-ES Algorithms with Surrogate Models and Line Searches. In Proceedings of the 15th annual conference companion on Genetic and evolutionary computation, pp. 1177–1184, 2013.
Anything else you would like to add:
I implemented CMA-ES in Python and Go.
- CyberAgent/cmaes is pure python CMA-ES implementation which only depends numpy.
- This library will be adopted by Optuna as its official sampler (see Add CMA-ES sampler. optuna/optuna#920).
- c-bata/goptuna provides pure Go CMA-ES implementation built on gonum/mat.