clean up upstream-dev CI

[keewis]

follow-up to #4583, which also added a PR upstream-dev CI. However, because the output of pytest has been redirected to a file, the failures are hard to diagnose. The fix is to duplicate stdout using tee.

Additionally, we now have two upstream-dev CI, so I guess we should remove the old one? #4584 will probably set the upstream-dev CI to allow_failure: true, so it might be better to remove the new CI (or to somehow mark the new PR CI with allow_failure: true while not changing the scheduled CI).

[andersy005]

@keewis, thank you for catching and addressing this...

[andersy005]

Looks good to me 👍

[keewis]

great, thanks again, @andersy005. Let's merge now and leave the choice between the old and new PR CI to a different PR.

[andersy005]

great, thanks again, @andersy005. Let's merge now and leave the choice between the old and new PR CI to a different PR.

Of course! I'm curious, are there any benefits to keeping the old azure upstream-dev CI around?

[keewis]

If I remember correctly, we decided to mark the upstream-dev CI as allowed failure. If there's a way to mark the PR CI as such, I think the only benefit is that a allowed failure of the old azure CI is a bit more visible (see #4584 (comment)).

[andersy005]

we decided to mark the upstream-dev CI as allowed failure. If there's a way to mark the PR CI as such,

If I understand this correctly, this is something that can be done via the branch protection settings from the repo settings.

[keewis]

That's somewhat different, I think. Branch protection defines CI that has to pass in order to be able to merge the PR, while allowed failures change the result of the CI. For the current implementation of this, see the changes in #4584.

Edit: the continue-on-error setting will do the same, except the job passes silently which is not ideal

[andersy005]

Oooh I see... Thank you for the clarification