Handling state_ids responses loads huge amounts of events into memory at once, causing OOMs

[richvdh]

For example: the response to matrix://matrix.org/_matrix/federation/v1/state_ids/%21jWRVQAlVGjigKCRGwS%3Amatrix.org?event_id=%241577162865947686RYsuB%3Amatrix.org includes an auth chain with 22327 events. surely there can't be that many events in $1577162865947686RYsuB:matrix.org's auth chain?

[richvdh]

I wonder if this has been caused (or exacerbated) by us deploying #6556

[richvdh]

right, so the problem here is that synapse is returning the auth_events for each of the events in the state of the room at the given event_id. This is consistent with the spec; however it seems to be utterly pointless. For a given event in the state, either:

• we already have it, and will therefore know its auth_events
• we don't have it, so will have to request it via /event which will tell us its auth events.

[richvdh]

we don't have it, so will have to request it via /event which will tell us its auth events.

unfortunately, as of #7817, synapse won't recursively fetch auth events in this way :/

[richvdh]

On reflection, it's correct that we return all these auth events. The problem is that we load them all into memory at once.

[richvdh]

this might be a bit better since #9601, but it's still problematic