`/joined_rooms returns only joined rooms` is flakey

[DMRobertson]

Probably another worker replication pain.

https://github.com/matrix-org/synapse/pull/11006/checks?check_run_id=3819561640

[DMRobertson]

Unable to reproduce with

docker run --rm -v /home/dmr/workspace/synapse:/src:ro -v /home/dmr/workspace/sytest/:/sytest -v /tmp/logs:/logs -e POSTGRES=1 -e WORKERS=1 -e MULTI_POSTGRES=1 matrixdotorg/sytest-synapse:testing --torture-replication 100 -s tests/30rooms/40joinedapis.pl

[DMRobertson]

Results.tap wasn't very illuminating:

not ok 306 /joined_rooms returns only joined rooms
# Started: 2021-10-06 19:51:33.246
# Ended: 2021-10-06 19:51:43.407
# Timed out waiting for test at /sytest/run-tests.pl line 787.
# 0.155834: Registered new user @anon-20211006_194652-279:localhost:8840
# 0.157143: Registered new user @anon-20211006_194652-280:localhost:8840
# 1.295316: room left
# "!edrqjLvxZuXgjPChKh:localhost:8840"
# 1.562932: room joined
# "!nfUcgGiGAYugOqoPMY:localhost:8840"

But looking at the test source it seems that the call to create $room_invited timed out (we never saw its log line).

Within the timeframe of the test, I can synapse logging as it completes its response to two calls to /createRoom. No sign of another until the next test has started. Can't see any warnings or errors during this timeframe.

Unclear if

• sytest never made the request
• it wasn't forwarded to synapse by haproxy; we don't have haproxy logs in the archive
• synapse didn't handle the request properly

Can take another look if we see this again.