[FEATURE] selector param works on `flagSetId` not `source`

[toddbaert]

Observed behavior

flagd supports selectors, so separate flags into different buckets, eg. each file source, http source, etc, is its own selector.
If flagd is started with multiple sources, the selector param can be used to evaluate/sync only the set of flags from the source in question:

Example

a.json and
b.json:

{
    "flags": {
        "flagb": {
            "state": "ENABLED",
            "variants": {
                "foo": "foo",
                "bar": "bar"
            },
            "defaultVariant": "foo"
        }
    },
    "metadata": {
        "flagSetId": "b"
    }
}

{
    "flags": {
        "flaga": {
            "state": "ENABLED",
            "variants": {
                "foo": "foo",
                "bar": "bar"
            },
            "defaultVariant": "foo"
        }
    },
    "metadata": {
        "flagSetId": "a"
    }
}

start flagd with those two files as sources:

docker run \
  --rm -it \
  --name flagd \
  -p 8015:8015 \
  -v $(pwd):/flags \
  ghcr.io/open-feature/flagd:latest start \
  --uri file:./flags/a.json \
  --uri file:./flags/b.json

for a.json

 grpcurl -import-path  <path-to-sync-proto-dir> -proto sync.proto -plaintext  -d '{"selector":"./flags/b.json"}'  localhost:8015 flagd.sync.v1.FlagSyncService/FetchAllFlags

and for b.json

grpcurl -import-path <path-to-sync-proto-dir> -proto sync.proto -plaintext  -d '{"selector":"./flags/a.json"}'  localhost:8015 flagd.sync.v1.FlagSyncService/FetchAllFlags

and for both (no selector):

grpcurl -import-path <path-to-sync-proto-dir> -proto sync.proto -plaintext  localhost:8015 flagd.sync.v1.FlagSyncService/FetchAllFlags

This is effective, however, it requires API users to have knowledge of the sources, and means that if a source location is changed, often, application code needs to be changed accordingly (brittle!).

Instead, the selector should work on a logical identifier for a flag set, the flagSetId, which can be maintained separately from the source.

Definition of Done:

• flagSetId is used to select flag sets, not the source
• updated documentation
• discuss whether or not this should be done in a non-breaking way

[tangenti]

I checked the code base and I have a two questions here

1. Since now we have the config and flag level metadata, and some of them are required, why can't we have key-value pairs as selector similar to what Kubernetes has?

We don't have to support all operations, k1=v1,k2=v2 sounds good enough as a start. Comparing to the current proposal, I think it prevents a breaking change (we could interpret the value alone as source=value). Plus, this can be extended easily when we want to support more attributes for selections.

2. selector is not a property of a flag, but of a gRPC sync request, however I can see that it's stored down to the in memory flags store. Any reason to that? Now it seems the only usage is to export the state of the store as a string.

[toddbaert]

Since now we have the config and flag level metadata, and some of them are required, why can't we have key-value pairs as selector similar to what Kubernetes has

None are required, some are specified in the schema so they are "first class" and have a semantic meaning, but they are optional.

why can't we have key-value pairs as selector

Do you mean selecting on things arbitrarily instead of just the flag set id?

selector is not a property of a flag, but of a gRPC sync request, however I can see that it's stored down to the in memory flags store. Any reason to that? Now it seems the only usage is to export the state of the store as a string.

Exporting the whole state as a string is how the in-process evaluation mechanism works. The selector is indeed used for this, so that only the flags relevant to that selector are returned. We also want to support something similar in RPC: #1611

[toddbaert]

Relevant slack discussion: https://cloud-native.slack.com/archives/C03J36ZP020/p1745250924025809

[juanparadox]

If flagd is started with multiple sources, the selector param can be used to evaluate/sync only the set of flags from the source in question

This is great, but what if you want the evaluation to occur at a lower level if you only know the set of flags to evaluate/sync lower down the chain? flagd -> provider?

This leads me to a question I have around the selector and how we might be able to provide a selector at the Provider level.

For example, if your flags support a more scoped set of flags by something like application and environment, would it be possible to configure the Provider in the constructor to point at a single set of flags based on a flagSetId or even an array (a number of) of flag set IDs?

---

Note: I did ask this in Slack, but figured we would want to discuss it further async here as to how the above might work.
Related conversation in slack: https://cloud-native.slack.com/archives/C03J36ZP020/p1745250924025809

[tangenti]

None are required, some are specified in the schema so they are "first class" and have a semantic meaning, but they are optional.

How could a selector for flagSetId work if it's not required in the config?

Do you mean selecting on things arbitrarily instead of just the flag set id?

No, I don't think it's a good idea to allow selecting on arbitrary things.
It's about the semantic of the selector value. Now it's only a string value representing the source, changing the interpretation to flagSetId is a breaking change, and any further extensions or change of the selector will be breaking because the syntax of the selector is not flexible at all.

Exporting the whole state as a string is how the in-process evaluation mechanism works. The selector is indeed used for this, so that only the flags relevant to that selector are returned. We also want to support something similar in RPC: #1611

Thanks for confirming. Given that, I think it's probably not a good idea to mix the selector into the flag storage. The evaluator should store the selectors used for different sources in a different way.