What are the train/test datasets when "validation_type": "split"?

[filipsPL]

Hi,
when the validation strategy is kfold, there is a folds folder generated with all folds. But where can I find such data for split strategy? (i.e., to see what was the training and testing dataset)?

I have the latest version of mljar-supervised (0.11.1).

best, Filip

[pplonski]

Hi @filipsPL,

looks like there is no such thing logged. Ups ... I've checked and for kfold we log indices in numpy format

mljar-supervised/supervised/validation/validator_kfold.py

Line 101 in 4181299

np.save(train_index_file, train_index)

Would it be ok for you @filipsPL to log it as numpy files?

[filipsPL]

Thanks for super-fast reply.

Yes, numpy format would be perfect!

[pplonski]

@MaciekEO please take a look at this issue. Please check:

• how it is saved in kfold validation

  mljar-supervised/supervised/validation/validator_kfold.py

  Lines 101 to 102 in 4181299

  	np.save(train_index_file, train_index)
  	np.save(validation_index_file, validation_index)

• it can be added in split validation, somewhere here

  mljar-supervised/supervised/validation/validator_split.py

  Line 85 in 4181299

  train_data = {"X": X_train, "y": y_train}

[pplonski]

Thanks @MaciekEO! @filipsPL the updates are already in the master branch. They will go to the next minor release.

You can install the package with updates:

pip install -q -U git+https://github.com/mljar/mljar-supervised.git@master

You will find the indices for train and validation in the AutoML results directory. The indices are saved in numpy format. You can read them like:

train_index = np.load(train_index_file)

@filipsPL please let us know if it works for you.

[filipsPL]

Great, it works! Thank you @pplonski @MaciekEO for quickly addressing the problem.