Surrogate model is only updated when point is optimal

[weslleyspereira]

Hi! I noticed a possible bug in your code for the DYCORS optimization method. Accordingly to the original paper,

Rommel G Regis and Christine A Shoemaker.
Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization.
Engineering Optimization, 45(5): 529–555, 2013.

the surrogate model should be updated regardless if it finds or not a new best point. So, the code:

Surrogates.jl/src/Optimization.jl

Lines 1047 to 1051 in b66096e

	if f_new < y_best
	x_best = x_new
	y_best = f_new
	add_point!(surr1, x_best, y_best)
	end

should be updated to:

if f_new < y_best
    x_best = x_new
    y_best = f_new
end
add_point!(surr1, x_new, f_new)

A similar fix applies to the multidimensional DYCORS optimization method.

[ChrisRackauckas]

Thanks, maybe we can make that an option and we can set the default to match what's original/standard here? Are you opening a PR?

[weslleyspereira]

Hi! I can make a PR.

I am not sure if the current behavior is useful in practice. If you find it is, could you please guide me on how I should add this new option to the routine?