Skip to content

Logs for ScheduledTellMsg DeadLetter are not specific enough #7264

@Arkatufus

Description

@Arkatufus

Version Information

Version of Akka.NET? dev branch
Which Akka.NET Modules? Core

Describe the bug

Dead letters containing ScheduledTellMsg from the Scheduler are hiding the underlying payload type it is wrapping, making it hard to debug/inspect via telemetry

Here's an example snippet log dump from one of the internal Akka.Cluster.Tools.Tests.Multinode.ClusterClient.ClusterClientDiscoerySpec test:

[WARNING][06/24/2024 14:51:03.026Z][Thread 0015][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.120Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.136Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.246Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.356Z][Thread 0016][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.464Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.571Z][Thread 0022][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.679Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.773Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.897Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.004Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>

Here a message sent from Self to Self via ScheduleTell (either once or repeated) was not being handled correctly, but we could not see which message was being sent, thus we could never tell which timer were at fault.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions