On exception during fit script doesn't terminate cleanly #1480
Description
Describe the bug
If an exception is triggered during the fit process of the auto-sklearn classifier the python interpreter process does NOT exit cleanly (needs a manual keyboard interrupt).
To Reproduce
Steps to reproduce the behavior:
- Write simple script to trigger exception
import autosklearn.classification
import pandas as pd
automl = autosklearn.classification.AutoSklearnClassifier()
a = pd.DataFrame(columns=["id"])
b = pd.DataFrame(columns=["id", "y"])
a.set_index("id", inplace=True)
b.set_index("id", inplace=True)
combined_data = pd.merge(a, b, left_index=True, right_index=True)
x = combined_data.drop('y', axis=1)
y = combined_data['y']
automl.fit(x, y)
- Execute script
python script.py - Script shows this backtrace:
Traceback (most recent call last):
File "script2.py", line 11, in <module>
automl.fit(x, y)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/estimators.py", line 1045, in fit
super().fit(
File "/usr/local/lib/python3.8/dist-packages/autosklearn/estimators.py", line 375, in fit
self.automl_.fit(load_models=self.load_models, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/automl.py", line 2056, in fit
return super().fit(
File "/usr/local/lib/python3.8/dist-packages/autosklearn/automl.py", line 639, in fit
self.InputValidator.fit(X_train=X, y_train=y, X_test=X_test, y_test=y_test)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/data/validation.py", line 147, in fit
self.target_validator.fit(y_train, y_test)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/data/target_validator.py", line 119, in fit
self._fit(y_train, y_test)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/data/target_validator.py", line 182, in _fit
self.encoder.fit(y_train.to_numpy().reshape(-1, 1))
File "/usr/local/lib/python3.8/dist-packages/sklearn/preprocessing/_encoders.py", line 779, in fit
self._fit(X)
File "/usr/local/lib/python3.8/dist-packages/sklearn/preprocessing/_encoders.py", line 77, in _fit
X_list, n_samples, n_features = self._check_X(
File "/usr/local/lib/python3.8/dist-packages/sklearn/preprocessing/_encoders.py", line 44, in _check_X
X_temp = check_array(X, dtype=None,
File "/usr/local/lib/python3.8/dist-packages/sklearn/utils/validation.py", line 63, in inner_f
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/sklearn/utils/validation.py", line 726, in check_array
raise ValueError("Found array with %d sample(s) (shape=%s) while a"
ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required.
Expected behavior
After showing the stack trace, script should terminate, I expect the python interpreter to quit.
Actual behavior, stacktrace or logfile
After showing the exception the script doesn't terminate, it's stuck processing some exit handlers.
If I send keyboard interrupt I get the following stack trace:
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 27, in poll
Process ForkProcess-1:
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/autosklearn/util/logging_.py", line 320, in start_log_server
receiver.serve_until_stopped()
File "/usr/local/lib/python3.8/dist-packages/autosklearn/util/logging_.py", line 350, in serve_until_stopped
rd, wr, ex = select.select([self.socket.fileno()],
KeyboardInterrupt
Environment and installation:
Please give details about your installation:
- Ubuntu 20.04, virtual env, python 3.10
- Docker (https://hub.docker.com/r/mfeurer/auto-sklearn/ which is same Ubuntu version), python 3.8
- auto-sklearn version 0.14.7