I am getting error in fit_ensemble() in 0.14.2 version

[shabir1]

fit_ensemble() method is not working in 0.14.2 version

I am getting error in fit_ensemble() method while using latest version of auto sklearn
I tried ensemble_size from 0, 10, 20, .. but same error

model.fit_ensemble(train_y.copy(), ensemble_size=ensemble_size)

Error: 
autosklearn/automl.py", line 1499, in fit_ensemble
    future = manager.futures.pop()
IndexError: pop from empty list

[eddiebergman]

Thanks for the bug report, there's three points here:

• This is already called in fit() if ensemble_size > 0, therefore there is no real need to call it again. This method essentially get's all models that were successful and attempts to build an ensemble out of them.
• We should raise an error if this is already done.
• This should be documented better.

Please refer to this example

[shabir1]

@eddiebergman What if ensemble_size=0 ?

[eddiebergman]

If ensemble_size = 0 in the __init__ params of autosklearn, no ensemble is fit, auto-sklearn just finds models and then completes. Calling fit_ensemble afterwards then accepts a param to how large of an ensemble you want, post fitting.

The example I linked to is actually run to produce the outputs there so it does work in that case.

[shabir1]

@eddiebergman Which example?

[eddiebergman]

Oh sorry I never gave the link in the comment above, here it is

[shabir1]

@eddiebergman I don't want the ensemble model.
I tried the below 2 ways but it failed both ways

automl = autosklearn.classification.AutoSklearnClassifier(
    time_left_for_this_task=120,
    per_run_time_limit=30,
    ensemble_size=0
)
automl.fit(X_train, y_train, dataset_name='breast_cancer')
automl.sprint_statistics()
automl.show_models()

[WARNING] [2021-12-27 12:14:01,525:Client-autosklearn.util.backend] Directory /tmp/autosklearn_tmp_0d28300d-66e0-11ec-bfa6-b365af5c825c/.auto-sklearn/ensembles does not exist
Dataset name: 0d28293e-66e0-11ec-bfa6-b365af5c825c\n  Metric: accuracy\n  Best validation score: 0.966667\n  Number of target algorithm runs: 67\n  Number of successful target algorithm runs: 67\n  Number of crashed target algorithm runs: 0
...
...
line 1542, in _load_models
    raise ValueError('No models fitted!')
ValueError: No models fitted!

automl = autosklearn.classification.AutoSklearnClassifier(
    time_left_for_this_task=120,
    per_run_time_limit=30,
    ensemble_size=0
)
automl.fit(X_train, y_train, dataset_name='breast_cancer')
model.fit_ensemble(train_y,  ensemble_size=0)
automl.sprint_statistics()
automl.show_models()

[WARNING] [2021-12-27 12:14:01,525:Client-autosklearn.util.backend] Directory /tmp/autosklearn_tmp_0d28300d-66e0-11ec-bfa6-b365af5c825c/.auto-sklearn/ensembles does not exist
line 1499, in fit_ensemble
    future = manager.futures.pop()
IndexError: pop from empty list
'''

Actually, I don't want an ensemble model at all

[shabir1]

@eddiebergman I don't want an ensemble model at all

[eddiebergman]

Sorry, if you set the ensemble size to 1 it should work. That is my bad. I will also document that an ensemble size of 0 should be allowed to be specified. The issue seems a bit convoluted as to why it doesn't work for show_models and is part of the rework.

I would dig a little deeper than show_models to get what you need, as the fitting procedure and sprint_statistics gave output without issue.