Make deletions via SCIM more stable

[supersven]

This story gave me some headache: I tried to make things better without adding new fields/tables or expensive queries / indexes. And, had to keep in mind that data, e.g. email addresses or handles may be reused by another account.

Changes:

• brig accounts are now deleted in a synchronous call (/i/users/:uid/ensure-deleted). This way the SCIM caller should at least get an error when something in brig went wrong.
• The endpoint can be re-called to ensure that a brig account is deleted. If there is still dangling data around, it re-runs the deletion.
• Some data may be re-used; e.g. handles or email addresses. These are deleted first. We cannot access them (without expensive queries / indexes) after the user tombstone has been written. So, the tombstone is written after they were deleted. (It has been like that before, however it's an important detail to keep / understand.)
• spar depends on user data from brig (veid) to delete it's user data. To ensure this is available when needed, spar data is deleted before brig data.

All this could have been much nicer without the query restrictions of Cassandra. 🤔

Checklist

• The PR Title explains the impact of the change.
• The PR description provides context as to why the change should occur and what the code contributes to that effect. This could also be a link to a JIRA ticket or a Github issue, if there is one.
• If HTTP endpoint paths have been added or renamed, or feature configs have changed, the endpoint / config-flag checklist (see Wire-employee only backend wiki page) has been followed. (Discussed in Backend channel.)
• changelog.d contains the following bits of information (details):
  • A file with the changelog entry in one or more suitable sub-sections. The sub-sections are marked by directories inside changelog.d.
  • If internal end-points have been added or changed: which services have to be deployed in a specific order?

[elland]

Looks good. I have a few nitpicky concerns about function names but that's not too important.

[fisx]

I've made two commits simplifying things, but I think there is more:

• can't we just modify the old Delete effect instead of removing it and adding a different one? semantics is still the same, no?
• similarly, can we modify the internal delete end-point in brig that was used before, rather than adding a new one?

while i was changing this, i found another instance where users are deleted on brig (namely when the idp is deleted and all the accounts in it are deleted with it). have you intentionally not updated that? why?

[supersven]

can't we just modify the old Delete effect instead of removing it and adding a different one? semantics is still the same, no?

@fisx , yes, your two commits regarding this seem to be perfectly reasonable. 👍

similarly, can we modify the internal delete end-point in brig that was used before, rather than adding a new one?

This question follows from your changes in spar (previously it was used by the function you deleted). It sounds like a good idea. 👍
As there's no response body in delete "/i/users/:uid", this change should be fine. (Operationally, the call would become synchronous instead of asynchronous. However, I cannot think of any issues regarding this.)

Speaking of synchronous vs. asynchronous: Haven you seen that the prior behavior was asynchronous (with an event) and this is now synchronous? Is that fine with you?

while i was changing this, i found another instance where users are deleted on brig (namely when the idp is deleted and all the accounts in it are deleted with it). have you intentionally not updated that? why?

I think I wasn't brave enough and a bit overwhelmed by the implicit relationships of side-effects. So, I kept my change set non invasive. Next time, I should be braver... 😄