use a custom error instead of calling `pytest.fail`

[12rambau]

Context

I'm currently building a pytest-regression spinoff (pytest-gee) dedicated to Google Earth Engine Python AP, the Google Earth Observation software. This as any Google product is just a monster API and the Python binding is a wrapper to build the correct json payload (earthengine-api). As a consequence my pytest plugin do 2 things, first check that the API call I'm building is still the same (by comparing yaml files) and if not, make a real call to the API to check the server response.

This reduce the computation time and cost of running the tests as the Google team is making lots of efforst to avoid non-regression.

In this context something like :

def test_list_regression(ee_list_regression):
    """Test the ee_list_regression fixture."""
    data = ee.List([1, 2, 3])
    ee_list_regression.check(data)

will create 2 files:

• /serialized_list_regression.yml
• /list_regression.yaml

the check on "serialized" is performed with perform_regression_check and I would like to silence anything that would happen there. basically if it works, good, if not I make a call to the API.

Issue

In the perform_regression_check method, you are directly calling pytest.fail when the file is not there and/or the file is regenerated. The problem for me is that pytest is not exposing the error type used by pytest.fail making it impossible to catch it is a supress context manager nor a try ... catch statement.

Could you explain why you didn't simply raised a fileNotFound error instead ? and if not clear reason would you be open to change this behavior ?

[nicoddemus]

Hi @12rambau!

The problem for me is that pytest is not exposing the error type used by pytest.fail making it impossible to catch it is a supress context manager nor a try ... catch statement.

Actually that is exposed as pytest.fail.Exception, so you can easily catch that.

Could you explain why you didn't simply raised a fileNotFound error instead ?

pytest.fail is more suitable in that case because then pytest treats it differently and does not show the full traceback.

[12rambau]

how did I failed (pun intended) to find this Exception in their documetnation ? Let me try on my end and I'll get back to you.

[nicoddemus]

It is not super prominent, being easy to miss: https://docs.pytest.org/en/stable/reference/reference.html#pytest.fail.Exception

[12rambau]

It works just fine in gee-community/pytest-gee#62 thanks for pointing me in the right direction !