User parameter data is referenced inside RelationalCommandCache

[renanhgon]

When I start my application, gradually, as processes are executed, the memory consumption just keeps increasing until my k8s pod simply dies due to lack of memory and gets restarted.

After analyzing the generated dumps, we observed the following:

To ensure that the dispose method of my DbContext is always executed, some logs were added during its creation and disposal.

public MyDbContext(IDatabaseConfig configuration, 
    ISeed seed, 
    ILogger<MyDbContext> logger)
{
    _contextId = Guid.NewGuid();
    _logger = logger;

    _logger.LogInformation("Context created. ID {0}", _contextId.ToString());

    _configuration = configuration;
    _seed = seed;
}

public override void Dispose()
{
    base.Dispose();
    _logger.LogInformation("Context destroyed. ID {0}", _contextId.ToString());
}

public override ValueTask DisposeAsync()
{
    var result = base.DisposeAsync();
    _logger.LogInformation("Context destroyed. ID {0}", _contextId.ToString());
    return result;
}

And from these logs, we were able to confirm that the object is indeed always created and disposed without any problems.

In some previous issues, I observed some possible causes within the OnConfiguring method, but I believe that is not the problem in my case.

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
    if (!optionsBuilder.IsConfigured)
        optionsBuilder.UseNpgsql(_configuration.ConnectionString);

    base.OnConfiguring(optionsBuilder);
}

My DbContext instance is resolved in this way:

private static void RegisterContext(IServiceCollection services) 
    => services.AddDbContext<IMyDbContext, MyDbContext>();

The issue was created here because Microsoft.EntityFrameworkCore.Internal.ServiceProviderCache is indicated. But perhaps the problem might be related to the GarbageCollector?

EF Core version: 8.0.6
Database provider: Npgsql.EntityFrameworkCore.PostgreSQL
Target framework: NET 8.0
Operating system: Windows 10
IDE: Visual Studio 2022 17.10.1
My container is running with this base image: mcr.microsoft.com/dotnet/aspnet:8.0

[ajcvickers]

This issue is lacking enough information for us to be able to fully understand what is happening. Please attach a small, runnable project or post a small, runnable code listing that reproduces what you are seeing so that we can investigate.

[renanhgon]

Sure! I created this example repository with an implementation very close to my real scenario.
https://github.com/renanhgon/EFCoreMemoryLeak

When I run my application, just the devops/ping route triggered by k8s is enough to cause a constant increase in memory usage. You can simulate these pings directly from the browser console:

setInterval(() => fetch('/v1/devops/ping'), 10000);
setInterval(() => fetch('/v1/devops/ping'), 30000);

[cssatkmd]

Has this issued been looked at or given any time and effort? We can reproduce a large memory usage, but we have yet to create a reproduceable example.
The example above seems to have a leak, so can this be confirmed or mitigated because of something?

[peterkiss1]

Something similar is happening in my app;

EF Core version: 8.0.6 or EF Core version: 8.0.4
Database provider: Npgsql.EntityFrameworkCore.PostgreSQL 8.0.3

The setup is simple as the database holds 3 tables, from which 2 are in relation in a one-to-many mapping.
Application is a Web API, web requests are interacting with the context but also hosted services are doing database operations, context registered as transient, hosted services are creating their scope for each process then dispose the whole, DbContext also disposed everywhere still the memory usage is sky rocketing (low load is on the app). PostgreSQL driver has an issue with multiplexing and OpenTelemetry but we are not using multiplexing.

[peterkiss1]

Executed a couple of extra tests and

• 8.0.0 looks better than 8.0.4 or 8.0.6
• if i turn ON pooling for the context everything looks fine which is bad as pooling itself is not an essential thing and i'm afraid that something deeper is simple wrong with EF

Who has memory issues please check the suffering apps with pooling.

[ajcvickers]

Note for team: we need to decide how to best investigate this kind of issue. My usual process doesn't work here, so I'd like to get advice from the team on how to proceed.

[ajcvickers]

@renanhgon Can you please post the OutOfMemoryException you get from running the repro until it runs out of memory?

[bleapleaper]

@ajcvickers Semi-similar issue where my stuff is kept in memory. Used PerfView (first time) to find that RelationalCommandCache uses my object in a cache key. No idea how it ended up there xD. However it would be nice if there was an option to turn on som debug messages when saving abnormal types to cache, wether MemoryCache or static

edit; never mind. memory cache says no entries but visual studio heap says two entries - no idea what is going on
edit2: oh its probably ef own service provider?

anyway ServiceProviderCache or some cache is holding back 1 million ...Core.Update.ColumnModification and 13000 entites from being garbage collected

[cliffankh0z]

@ajcvickers found the bug, inspect IMemoryCache in a breakpoint and you'll see what is mean. Lists of Guids and all different kinds of parameter values are put in IMemoryCache as CommandCacheKey. Only need the type info for the cache key, not the actual parameter value. These object can link to other object, which links to other objects thereby keeping all of them in memory

Youll have to use optionsBuilder.UseMemoryCache(_memoryCache); to be able to inspect this