[Host.Redis] Redis consumer doesn't process events after reconnect

[cyril265]

Hello

the issue appears when the consumer app briefly loses the connection to redis. After that no events are processed by the consumer until you restart it.

to reproduce:

• start producer and consumer
• send an event (works)
• close and open the redis connection
• send an event - the consumer won't consume/process the event anymore until you restart it

Normally RedisListCheckerConsumer prints logs like this if it's working:

[11:01:52 VRB] Checking keys...
[11:01:52 VRB] Performing delay since no new items arrived

After the disconnect/reconnect RedisListCheckerConsumer doesn't log anything

producer:

    builder.Services.AddSlimMessageBus(mbb => {
        mbb.WithProviderRedis(settings =>
            settings.ConnectionString = "<connectionString>";
        mbb.AddServicesFromAssembly(Assembly.GetExecutingAssembly());

        mbb.Produce<TenantCreated>(producerBuilder => { producerBuilder.DefaultQueue("tenantCreated"); });

        mbb.AddJsonSerializer();
    });

consumer:

    builder.Services.AddSlimMessageBus(mbb => {
        mbb.WithProviderRedis(settings =>
            settings.ConnectionString = "<connectionString>";
        mbb.AddServicesFromAssembly(Assembly.GetExecutingAssembly());

        mbb.Consume<TenantCreated>(producerBuilder => { producerBuilder.Queue("tenantCreated");   });

        mbb.AddJsonSerializer();
    });

Version: 3.3.0

[zarusz]

Perhaps a reconnect for transient network issues is required here. I think such a retry feature was requested, but not sure if it was implemented yet (no access to computer right now). I suppose this is not a new bug. We may need to look into this. Happy to accept a PR, if you wanted to expedite this before I get to this.

Also, Redis does not store the messages if the consumer were to disconnect, the messages are passed and replicated to active consumers only.

[cyril265]

I'm not sure it's a reconnect issue since the producer resumes working correctly once the connection is restored. From what I understand, IConnectionMultiplexer handles reconnections automatically.

The issue seems to be that RedisListCheckerConsumer stops executing for some reason. It's possible that OnStop is being triggered or something similar is happening.

I'm definitely open to submitting a pull request. Could you please point me to a good starting place?

[zarusz]

Hi @cyril265, after re-reading the issue, I realized I initially overlooked that the problem involves the RedisListCheckerConsumer, which emulates queues using Redis lists.

You're absolutely right the underlying client does reconnect automatically already - but it’s likely that when Redis temporarily loses connectivity, and before the underlying client reconnects, the main loop in RedisListCheckerConsumer crashes without any log output. This probably happens when a transient network failure causes an exception at this line:

SlimMessageBus/src/SlimMessageBus.Host.Redis/Consumers/RedisListCheckerConsumer.cs

Line 72 in e0c1d59

var value = await _database.ListLeftPopAsync(queue.Name).ConfigureAwait(false);

A good improvement would be to wrap this call in a try/catch block to handle transient exceptions, log a warning, and retry the pop operation with a delay.

Would you be interested in putting together a fix? Feel free to open a PR with the change and include a test to cover the scenario.

Thanks again!