add feature importance heatmap for all models

[aplonska]

[aaryanshsahay]

Hi , I'd like to contribute. Can you elaborate a bit?

[aplonska]

Hi, @aaryanshsahay thanks for your response! It will be nice to have one plot that will show feature importance from all models as a heatmap. The heatmap should be in the main README.md file. @pplonski could you provide implementation details?

[pplonski]

hey @aaryanshsahay!

The heatmap should be for all models. There can be easily much more than 13 models.

At the end of the model training, the documentation is created. Each model has its own directory. In this directory, there are many files with different information. If permutation-based importance is computed (explain_level > 0) then the feature importance plot is created. The example plot is below:

In the model directory, you can find files with importance computed. They will have the format: learner_fold_{*}_importance.csv. They are plotted and saved in the permutation_importance.png file.

But it is a plot just for one model. The idea is to take all importance for all models and plot them as a heatmap.

This should be done at the end of AutoML training (all models should be trained). If there will be only one model then the plot should be skipped.

[aaryanshsahay]

Hey @pplonski ! I understand this but I'm a beginner and I think that this is a little bit out of my scope , Thank you for taking time and explaining but i'm afraid i wont be able to complete this task , I apologize if I wasted your time.

[pplonski]

@aaryanshsahay no problem!

[prachipancholi]

I am unable to find this error in repo. Can you help?

[pplonski]

Hi @prachipancholi! The importance plots are done in supervised.utils.additional_metrics.py. There is add_permutation_importance() method. This method is reading feature importance files for a single model.

[pplonski]

@prachipancholi one more thing, it is not a bug. It is a new feature to be added. The first step would be to load importance from all models and then plot them as heatmap.

[prachipancholi]

@pplonski yes, thanks for reply. I got it.