-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Labels
Milestone
Description
Version Information
Version of Akka.NET? dev branch
Which Akka.NET Modules? Core
Describe the bug
Dead letters containing ScheduledTellMsg from the Scheduler are hiding the underlying payload type it is wrapping, making it hard to debug/inspect via telemetry
Here's an example snippet log dump from one of the internal Akka.Cluster.Tools.Tests.Multinode.ClusterClient.ClusterClientDiscoerySpec test:
[WARNING][06/24/2024 14:51:03.026Z][Thread 0015][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.120Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.136Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.246Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.356Z][Thread 0016][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.464Z][Thread 0017][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.571Z][Thread 0022][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.679Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.773Z][Thread 0020][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:03.897Z][Thread 0019][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.004Z][Thread 0018][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
[WARNING][06/24/2024 14:51:04.112Z][Thread 0021][akka://ClusterClientDiscoverySpec/user/$b] DeadLetter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490] to [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: <Received dead letter from [akka://ClusterClientDiscoverySpec/user/$b#1239270490]: Akka.Actor.Scheduler.ScheduledTellMsg>
Here a message sent from Self to Self via ScheduleTell (either once or repeated) was not being handled correctly, but we could not see which message was being sent, thus we could never tell which timer were at fault.