e2e tests follow up

[hassnian]

Let's add it to the unit/e2e testing for every bug-fix PR, if possible. I think bug fixes that do not require chain interaction are easier to include in the unit/e2e testing

Originally posted by @preschian in #11295 (comment)