Generation of Dapr-specific manifest resources

[philliphoff]

Customizes the generation of manifest resources for Dapr (sidecar) resources, exposing configured options, which is a prerequisite for tools to deploy Dapr-enabled Aspire applications (such as Azure Dev CLI to deploy to Azure Container Apps).

Dapr Component Resources

This change includes the addition of a new "Dapr component" resource that represents an individual Dapr component, such as a state store or pub-sub.

Components can be specified explicitly, for example:

var myStateStore = builder.AddDaprComponent("my-state-store", "state.redis");
var myPubSub = builder.AddDaprComponent("my-pub-sub", "pubsub.redis");

Alternatively, components can be specified in "generic form", for example:

var myStateStore = builder.AddDaprStateStore("my-state-store");
var myPubSub = builder.AddDaprPubSub("my-pub-sub");

The generic form is useful in that it allows Aspire (and/or deployment tooling) to automatically provision and configure an appropriate Dapr component for the generic type. When running locally, for example, the following components will be initialized:

Building Block	Dapr Initialized?	Provisioned Component
State Store ("state")	Full	Redis State Store
	Slim	In-Memory Store
Pub-Sub ("pubsub")	Full	Redis Pub-Sub
	Slim	In-Memory Pub-Sub

⚠️ Note that the in-memory pub-sub component can only receive messages published by that same application.

Options (such as the path to the local configuration file for the component) can also be specified, for example:

var myStateStore = builder.AddDaprComponent("my-state-store", "state.redis", new DaprComponentOptions { LocalPath = "<path>" });

Dapr component resources can then be associated with one or more individual projects (that are also associated with Dapr sidecars). In doing so, Aspire will automatically configure the appropriate sidecars to load those components (rather than require explicitly setting the "resources paths" of the sidecar).

For example:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddDapr();

var stateStore = builder.AddDaprStateStore("statestore");
var pubSub = builder.AddDaprPubSub("pubsub");

builder.AddProject<Projects.DaprServiceA>("servicea")
       .WithDaprSidecar("service-a")
       .WithReference(stateStore)
       .WithReference(pubSub);

builder.AddProject<Projects.DaprServiceB>("serviceb")
       .WithDaprSidecar("service-b")
       .WithReference(pubSub);

using var app = builder.Build();

await app.RunAsync();

Dapr Component Resource Manifest JSON

Dapr components are written to the manifest as individual resources of type dapr.component.v0 with specific options in a daprComponent object.

{
  "resources": {
    "<name>": {
      "type": "dapr.component.v0",
      "daprComponent": {
        "localPath": "...",
        "type": "..."
      }
    }
  }
}

Note: the use of a daprComponent sub-object is mostly for the benefit of Go-based tooling deserialization (such as Azure Dev CLI), where the various resource type properties might otherwise collide.

Dapr Sidecar Resource Manifest JSON

Dapr sidecars are written to the manifest as individual resources of type dapr.v0 with specific options in a dapr object. Of note are the application and components properties, which store references to the project associated with the sidecar and all the components associated with that project, respectively. Note also that external tools (e.g. deployment tooling) may only use a subset of options, depending on the scenario (e.g. resourcesPath and runFIle are both not applicable to Azure Container Apps).

{
  "resources": {
    "<name>": {
      "type": "dapr.v0",
      "dapr": {
        "application": "...",
        "appChannelAddress": "...",
        "appHealthCheckPath": "...",
        "appHealthProbeInterval": 0,
        "appHealthProbeTimeout": 0,
        "appHealthThreshold": 0,
        "appId": "...",
        "appMaxConcurrency": 0,
        "appPort": 0,
        "appProtocol": "...",
        "command": [ "...", "..." ],
        "components": [ "...", "..." ],
        "config": "...",
        "daprGrpcPort": 0,
        "daprHttpMaxRequestSize": 0,
        "daprHttpPort": 0,
        "daprHttpReadBufferSize": 0,
        "daprInternalGrpcPort": 0,
        "daprListenAddresses": "...",
        "enableApiLogging": true,
        "enableAppHealthCheck": true,
        "logLevel": "...",
        "metricsPort": 0,
        "placementHostAddress": "...",
        "profilePort": 0,
        "resourcesPath": [ "...", "..." ],
        "runFile": "...",
        "unixDomainSocket": "..."
      }
    }
  }
}

Note: the use of a dapr sub-object is mostly for the benefit of Go-based tooling deserialization (such as AZD CLI), where the various resource type properties might otherwise collide.

Future work

In cases where no existing Dapr-provisioned state store or pub-sub exists (e.g. dapr init --slim), rather than use the limited in-memory components, Aspire might instead explicitly provision a Redis (or similar) container. This makes additional assumptions, however, such as the existence of Docker host.

Related to:

• Design manifest output for dapr sidecar support #371
• ACA Deployment with Dapr #838

[davidfowl]

This will be problematic depending on where you run from. I wonder if there's anything we can do here to make this stable. Maybe an env variable in launch settings?

[philliphoff]

The intent was that component file paths be relative to the referenced project file (directory), which would be similar to how paths are expressed when using Dapr run files. That still could problematic if the same component is referenced by multiple projects but where those projects are not at the same relative location in the solution. A environment/configuration value might be ok...I'll think about it a bit.

[davidfowl]

We made a fix in my latest PR. You can get the apphost project directory from the builder so it's stable no matter where you run from. This can be a follow up PR.

[davidfowl]

This PR is looking pretty complete. Let's try to get this in today or tomorrow @philliphoff.

[davidfowl]

@philliphoff Any paths specified need to be resolved as relative to the manifest. There's a new helper for that btw.

[philliphoff]

Any paths specified need to be resolved as relative to the manifest. There's a new helper for that btw.

I'll take a look at that.

[davidfowl]

Also see 098c33b

[mitchdenny]

It looks like you've got an overloaded version of the WithReference extension method. Given you are doing that I'm wondering whether we need the WithDaprSidecar call? Couldn't your custom implementation of `WithReference check to see that the annotation is present (and could the name be derived?).

[philliphoff]

Do you mean that, when referencing components, that the need for the sidecar can be inferred?

[mitchdenny]

Yeah that is what I was thinking. Just removing a bit of (potentially) unnecessary ceremony?

[mitchdenny]

It might not be necessary in the case of Dapr, but I am wondering whether we should be creating a Aspire.Dapr package for the service side like we do with other Aspire components. For example should we be automatically wiring up some telemetry and healthchecks for dapr interactions within the service code for consistency.

Adding @eerhardt for his thoughts on this.

[philliphoff]

I think there are definitely opportunities to create Aspire components specific to Dapr, that wrap the Dapr .NET SDK is ways that make sense to tie into Aspire features (service discovery, perhaps?). I haven't had an opportunity to look closely at Aspire components and what it really means to create one. For example, Dapr already ties into Open Telemetry at the sidecar level, which is ultimately doing everything on behalf of the application, so I don't know if there would be anything specific to wrap in the SDK.

[mitchdenny]

OK ... we don't need to solve it in this PR I don't think given Dapr already integrates very well with DI.

[eerhardt]

We discussed in the past having "platform components"

#242
#243

It would seem this would fit in with that.

[mitchdenny]

@davidfowl I'm inclined to approve this PR now and get it in so we can work with the deployment tool folks to see if we can get this working end to end.

[philliphoff]

Any paths specified need to be resolved as relative to the manifest. There's a new helper for that btw.

@davidfowl @mitchdenny Is there any assumption that the manifest is always generated in the app host directory (as seems to be the case by default, anyway). If not, then that would mean relative paths specified in the Program.cs intended to be relative to the app host directory, would then need to be converted to a path relative to the manifest directory?

I guess the best practice would be for users to always pass full paths (based on IDistributedApplicationBuilder.AppHostDirectory) to Aspire APIs so that no specific conversion logic is needed.

[mitchdenny]

Paths in the manifest are relative to the manifest file. We just recently updated the API surface for manifest generation to have a context which exposes a GetManifestRelativePath(...) call to make this easier.

[philliphoff]

Paths in the manifest are relative to the manifest file. We just recently updated the API surface for manifest generation to have a context which exposes a GetManifestRelativePath(...) call to make this easier.

Right, but say a developer specifies a relative path in an options object. When using Aspire to run the app, it seems reasonable for that path to be relative to the app host directory. However, when using Aspire to generate the manifest, and that manifest is configured to be placed in a different directory than the app host, that relative path no longer works; it would need to be converted. While that's certainly possible, the places where manifest writing happens are part of annotations and not necessarily in a place to easily acquire the app host directory from the DI container.

I wonder if the GetManifestRelativePath() method could/should do that conversion automatically. That is, all paths specified in Aspire be considered relative to the app host and it does the conversion to be relative to the manifest path (should it be different than the app host).

Or, just tell developers to not use relative paths in Aspire APIs which avoids the issue.

[davidfowl]

However, when using Aspire to generate the manifest, and that manifest is configured to be placed in a different directory than the app host, that relative path no longer works; it would need to be converted.

This should still work. With your current code it'll work since you are first resolving paths relative to the apphost, then using GetMainfestRelativePath.

I wonder if the GetManifestRelativePath() method could/should do that conversion automatically. That is, all paths specified in Aspire be considered relative to the app host and it does the conversion to be relative to the manifest path (should it be different than the app host).

It can, and will. We can merge this PR 😄