Messages sent to the DLQ topic are not counted in micrometer metrics.

[Annlazar]

In case of the failure-strategy set to dead-letter-queue, if there is an exception while processing the message, the message can be consumed from the dead-letter queue topic, as described in https://quarkus.io/blog/kafka-failure-strategy/

But in the MicroMeter registry, the error is not counted, only in the incoming channel:
"mp.messaging.message.count;channel=ping-events": 1.0,
"mp.messaging.message.count;channel=pong-events": 0.0,
"mp.messaging.message.count;channel=dead-letter-topic-ping-events": 0.0,

The content of the header of the message consumed from DLQ:
dead-letter-exception-class-name: java.lang.IllegalArgumentException
dead-letter-reason: java.lang.IllegalArgumentException
dead-letter-topic: ping-events
dead-letter-partition: 0

The dead-letter-topic events should be counted in micrometer.

Java version: 17, Maven: 3.9.5, quarkus-smallrye-reactive-messaging-kafka, quarkus-micrometer-registry-prometheus: 3.2.9.Final-redhat-00003

[vietk]

Hello,

I wanted to bring up the importance of having metrics implemented for dead letter queue topic. Monitoring the activity and health of DLQs through metrics can provide insights into the health of our system and help in troubleshooting our message processing pipeline, here are a few key metrics that I believe would be beneficial to track:

• Message Count: Monitoring the number of messages being routed to the dead letter queue over time can give us a good indication of any issues occurring in our system.
• Message recovery: messages that goes to the dead letter queue often need manual recovery, like fixing out of band messages inside a specific flow can lead. With a proper alerting we can improve the recovery time.

Looking forward to your feedback.
Regards

[ozangunalp]

This was addressed in #2565 as part of the Message Observation API. In Quarkus integration this is backed by the Micrometer extension.

The Kafka DLQ strategy creates an internal channel named [incoming-channel-name]-dead-letter-queue and decorates the channel stream with outgoing decorators (SubscriberDecorator).