[bug]: Memory leak during watch on reconnection #824
Description
Describe the bug
At seemingly random times during both local debugging and a deployed operator I receive the Watcher for MyEntity was terminated and is reconnecting message. Code here. This occurs due to the known error when no resources exist for the entity type. Debug logs show this code being the source of the log statement.
This issue is that each time this occurs there appears to be a subtle memory leak, noted by Visual Studio snapshot diffs, and just an increase in memory over time for a deployed operator. VS diagnostics seemed to point the finger at yamldotnet attributes but I don't know enough to be certain.
I've had a really basic operator start out at 28Mb and leave it overnight with it sitting at 60+Mb, which is dependent on how many of these exceptions occurred.
To reproduce
Not certain how to forcefully replicate the exception that occurs to initiate an iteration of the watcher loop but assuming you can do that then that should be all that is required to replicate. You don't need controllers to have any behaviour, and only the CRDs need to be deployed, and no manifests.
Expected behavior
Expect no memory leak after an iteration of the watch loop, or at least for the memory to eventually be collected by GC.
Screenshots
No response
Additional Context
No response