Traceback on background process 'user_directory.notify_new_event' threw an exception

[jpds]

Description

I just upgraded a homeserver from 1.26 to 1.27 and now I'm seeing a lot of these tracebacks in my homeserver.log:

2021-02-16 17:36:09,420 - synapse.metrics.background_process_metrics - 211 - ERROR - user_directory.notify_new_event-9 - Background process 'user_directory.notify_new_event' threw an exception
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/metrics/background_process_metrics.py", line 208, in run
    return await maybe_awaitable(func(*args, **kwargs))
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/user_directory.py", line 110, in process
    await self._unsafe_process()
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/user_directory.py", line 159, in _unsafe_process
    self.pos, room_max_stream_ordering
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/databases/main/state_deltas.py", line 51, in get_current_state_deltas
    prev_stream_id = int(prev_stream_id)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType

Version information

• Homeserver: Private
• Version: 1.27
• Install method: Debian package.
• Platform: Debian

[toushin-taishi]

I am also experiencing the same issue when I upgraded a homeserver running on Ubuntu 18.04 using DEB packages to 1.27.

[Softace42]

A similar problem after upgrading from 1.26.0 to 1.27.0 (debian package). Ubuntu server 20.04

2021-02-17 15:47:16,958 - synapse.metrics.background_process_metrics - 210 - ERROR - user_directory.notify_new_event-22 - Background process 'user_directory.notify_new_event' thr>
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/metrics/background_process_metrics.py", line 208, in run
    return await maybe_awaitable(func(*args, **kwargs))
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/handlers/user_directory.py", line 110, in process
    await self._unsafe_process()
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/handlers/user_directory.py", line 158, in _unsafe_process
    max_pos, deltas = await self.store.get_current_state_deltas(
  File "/opt/venvs/matrix-synapse/lib/python3.8/site-packages/synapse/storage/databases/main/state_deltas.py", line 51, in get_current_state_deltas
    prev_stream_id = int(prev_stream_id)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[mithodin]

yup, same here.

[mithodin]

The direct cause seems to be (for me) that the stream_id column in the user_directory_stream_pos table is empty.

[maranda]

Same happened here, upgrading from 1.26 to 1.27

[maranda]

UPDATE: actually rebuilding the user directory index (see: https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md) seemingly fixed the tracebacks.

[clokep]

This seems to be a regression from https://github.com/matrix-org/synapse/pull/9223/files#r564010797.

[clokep]

UPDATE: actually rebuilding the user directory index (see: https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md) seemingly fixed the tracebacks.

This seems like a reasonable workaround! 👍 I'll put up a fix shortly which should go into v1.28.0 (to go to RC tomorrow).