Scope Ollama resilience settings to Web/Program.cs and restore ServiceDefaults

[ViveliDuCh]

Fixes #6331

Restore standard global HTTP client resilience configuration in ServiceDefaults/Program.cs, removing Ollama-specific timeouts and settings. Apply extended timeouts and custom resilience handlers only to chat_httpClient and embeddings_httpClient in Web/Program.cs using RemoveAllResilienceHandlers and AddStandardResilienceHandler. Scope Ollama configuration to the web project to prevent unintended global changes and align with .NET Aspire template conventions.

Microsoft Reviewers: Open in CodeFlow

[jozkee]

how are these named http clients used by ollama?

[jozkee]

As we discussed offline, this is not ideal as chat_httpClient is an implementation detail, we shouldn't depend on it.

[MackinnonBuck]

Agreed, it would be nice to not rely on the internal behavior of another library like this.

That said, I don't currently see another option. We need a way to get the IHttpClientBuilder. This might be a feature request for https://github.com/CommunityToolkit/Aspire.

[jozkee]

I don't currently see another option

I think the intent of #6331 was to just move the following to Web/Program.cs

extensions/src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.ServiceDefaults/Extensions.cs

Lines 33 to 42 in c4e57fb

	// Turn on resilience by default
	http.AddStandardResilienceHandler(config =>
	{
	// Extend the HTTP Client timeout for Ollama
	config.AttemptTimeout.Timeout = TimeSpan.FromMinutes(3);
	// Must be at least double the AttemptTimeout to pass options validation
	config.CircuitBreaker.SamplingDuration = TimeSpan.FromMinutes(10);
	config.TotalRequestTimeout.Timeout = TimeSpan.FromMinutes(10);
	});

And remove that #if (IsOllama) block, @MackinnonBuck, wouldn't that suffice? As per @jongalloway, that would narrow it down to configure only the clients in Web.csproj.

We would have two ConfigureHttpClientDefaults calls, though, not sure if that's OK.

[jozkee]

Additionally, we tried to narrow it down further checking the IHttpClientBuilder names:

builder.Services.ConfigureHttpClientDefaults(http =>
{
    if (http.Name.Contains("chat") || http.Name.Contains("embeddings"))
    {
#pragma warning disable EXTEXP0001 // RemoveAllResilienceHandlers is experimental
        http.RemoveAllResilienceHandlers();
#pragma warning restore EXTEXP0001

        // Turn on resilience by default
        http.AddStandardResilienceHandler(config =>
        {
            // Extend the HTTP Client timeout for Ollama
            config.AttemptTimeout.Timeout = TimeSpan.FromMinutes(3);

            // Must be at least double the AttemptTimeout to pass options validation
            config.CircuitBreaker.SamplingDuration = TimeSpan.FromMinutes(10);
            config.TotalRequestTimeout.Timeout = TimeSpan.FromMinutes(10);
        });

        // Turn on service discovery by default
        http.AddServiceDiscovery();
    }
});

But this didn't work, we only observed builders with null names. We would need to dig deeper to understand what's happening, just letting you know in case it rings a bell.

[jongalloway]

Agree, my main request is that we scope to the web project as opposed to the entire Aspire solution. Could also add a comment explaining the behavior and recommending removing the block of code before deploying to production (since Ollama won't be used in production).

[MackinnonBuck]

Ah, got it. Thanks for the clarification.

My main suggestion would be to consider putting this logic somewhere other than Program.cs - maybe another file defining an additional extension method that gets called from Program.cs.