Thoughts on Microsoft.Extensions.AI

[bugproof]

https://github.com/dotnet/extensions/tree/main/src/Libraries/Microsoft.Extensions.AI

It seems like the goal is similar no?

Looks like they support only:

• Amazon bedrock via AWSSDK.Extensions.Bedrock.MEAI package. I don't know if it supports anthropic - it should
• OpenAI via Microsoft.Extensions.AI.OpenAI pre-release package

AFAIK official Anthropic API is not supported via those abstractions yet.

LlmTornado seems like way ahead of that official Microsoft lib.

Looks like none support Batching (50% off offering) from anthropic and openai..

[lofcz]

Not familiar with Microsoft.Extensions.AI, but at a first glance, yes, it seems somewhat similar. Apart from the scope of supported APIs, I'd say the other difference is that we ship our own Toolkit with higher-level abstractions, and even in the core library, we have abstractions such as Conversation (Memory).

As for batching, we currently don't support it either - see Feature Matrix, but I'm open to either accepting a PR or implementing that myself, if no one picks it up.

Also, Tornado is impartial and has no incentives to prioritize shipping features from some Providers over others (which you may or may not see as a benefit).

[luisquintanilla]

Hi @lofcz,

This is a really amazing project. Great work!

I'm part of the team behind the AI and Vector Data Extensions which we shipped as GA a few weeks ago.

https://devblogs.microsoft.com/dotnet/ai-vector-data-dotnet-extensions-ga/

To add a bit of context on what these extensions are and how they’re being used:

Tornado is impartial and has no incentives to prioritize shipping features from some Providers over others

As mentioned in the post, the AI and Vector Data Extensions provide shared abstractions and utilities for working with AI models and vector stores. The goal is to promote interoperability and extensibility, which aligns closely with what LlmTornado is aiming to achieve. You might also notice today the abstractions are used by a variety of official and community NuGet packages beyond the ones mentioned in the original post.

we ship our own Toolkit with higher-level abstractions, and even in the core library, we have abstractions such as Conversation (Memory).

That’s fantastic! It's exactly the kind of scenario these extensions are designed to support. They’re meant to serve as a foundational layer for higher-level tooling and frameworks. Projects like MCP, Semantic Kernel, and Progress Telerik, are just a few examples of that.

https://devblogs.microsoft.com/dotnet/ai-vector-data-dotnet-extensions-ga/#a-growing-ecosystem

These abstractions act as the glue and help different systems interoperate in a standardized and consistent way.

We’d be happy to collaborate with you on integrating the AI and Vector Data Extensions into LlmTornado to help enable smoother interoperability across the broader .NET AI ecosystem.

cc: @stephentoub @jeffhandley @JeremyLikness

[lofcz]

Thanks for reaching out and explaining goals of the library in question. If I understand your statement correctly, the last paragraph suggests we should add a dependency on Extensions and rework our architecture to implement the abstractions defined in Extensions?

[luisquintanilla]

we should add a dependency on Extensions and rework our architecture to implement the abstractions defined in Extensions

That could be a way to go about it.

Looking at some of your Chat, Embedding and VectorStores types, there are a few that could be substituted by the extensions.

Here's a full list of types the extensions provide: https://learn.microsoft.com/dotnet/api/microsoft.extensions.ai?view=net-9.0-pp

Another way you might also consider is shipping as an adapter package which maps your types to Extensions.AI types. Similar to: https://github.com/rabuckley/GeminiDotnet

As mentioned, we're here to help and happy to chat through options with you.

[lofcz]

This seems like a significant undertaking on our part, as it adds a dependency we don't need, given that we already have all the necessary infrastructure in place. It also means losing control over our own abstractions and extending Microsoft's influence over .NET AI integrations. I'd appreciate understanding the strategic benefits that would make this worthwhile for the project.

[luisquintanilla]

...seems like a significant undertaking on our part, as it adds a dependency we don't need, given that we already have all the necessary infrastructure in place.

That's fair. To clarify, it's not all-or-nothing with the extensions. You can choose to opt-in to as little or as much of the abstractions as you want. As I mentioned before, we're happy to help out here.

I'd appreciate understanding the strategic benefits that would make this worthwhile for the project.

A few scenarios I can think of.

• The MCP C# SDK leverages the extensions, so your users would have an easier time using MCP tools.

  https://github.com/luisquintanilla/AccedeSimple/blob/00e8e41dd083b3f00f5a2fae37e76be69de3e0c9/src/AccedeSimple.Service/ProcessSteps/TravelPlanningStep.cs#L92-L101

• Users who want to build agents using libraries like Semantic Kernel can more easily plug LLM Tornado as the LLM provider.

  https://gist.github.com/luisquintanilla/931bca3f8f1f342983c6e0b06a41f96e

• Users who want to run evaluations can more easily leverage the AI Evaluations library.

  https://github.com/luisquintanilla/AccedeSimple/blob/main/AccedeSimple.EvalTests/RetrievalEvaluation.cs

[lofcz]

@luisquintanilla thanks for clarifying the benefits. Tornado now supports the official .NET MCP SDK via LlmTornado.Mcp adapter: https://github.com/lofcz/LlmTornado/blob/master/src/LlmTornado.Mcp.Sample/Program.cs