[FEATURE] ⚙️ Add `DefaultEntryOptionsProvider`

[jodydonetti]

Problem

Not really a "problem", but more like a potential new feature: as emerged in #367 there may be the need to not have only one single DefaultEntryOptions, but allow users to centrally specify a custom logic that can provide different default entry options based on the cache key that it is being worked on.

Solution

Important

NOTE: nothing is confirmed yet. As of now I'm playing with the idea, but I still need to implement it and see what happens in all edge cases, etc.

The idea is to add a new option in the FusionCacheOptions class so that it's possible to specify such custom provider logic.

The initial name here #367 was "builder", but that may indicate a way to build entry options everytime, which is not necessarily true: therefore a better name feels like "provider", because it is a way to provide default entry options based on some custom logic.

The current shape is:

public class FusionCacheOptions {
  // ...
  public Func<EntryOptionsProviderContext, FusionCacheEntryOptions?> DefaultEntryOptionsProvider { get; set; }
}

The context object contains the needed informations to implement the custom logic, and the shape is:

public class EntryOptionsProviderContext
{
	public string Key { get; }
	public FusionCacheEntryOptions DuplicateDefaultEntryOptions();
}

Two things to consider:

• of course if the custom logic is not specified, everything remains the same as before
• the custom logic can return null, as a way to express "in this case I'm good with the normal DefaultEntryOptions"

This means that the custom logic can:

• create a new FusionCacheEntryOptions from scratch, and return it
• duplicate the DefaultEntryOptions, apply some logic, and return it
• return null

This would allow customizations such as these:

// 1. BASIC:
// - ALWAYS CREATE A NEW ONE
DefaultEntryOptionsProvider = ctx => {
  return new FusionCacheEntryOptions().SetFailSafe(...);
};

// 2. MORE COMPLEX:
// - IF KEY STARTS WITH "foo/" -> CREATE A NEW ONE
// - OTHERWISE JUST RETURN THE DEFAULT ONE
DefaultEntryOptionsProvider = ctx => {
  if (ctx.Key.StartsWith("foo/")) {
    // CREATE A NEW ONE
    return new FusionCacheEntryOptions().SetFailSafe(...);
  }
  
  // RETURN NULL (USES THE DEFAULT ONE)
  return null;
};

// 3. MOST COMPLEX:
// - IF KEY STARTS WITH "foo/" -> CREATE A NEW ONE
// - IF KEY STARTS WITH "bar/" -> DUPLICATE THE DEFAULT ONE AND CHANGE SOMETHING
// - OTHERWISE JUST RETURN NULL (USES THE DEFAULT ONE)
DefaultEntryOptionsProvider = ctx => {
  if (ctx.Key.StartsWith("foo/")) {
    return new FusionCacheEntryOptions().SetFailSafe(...);
  }
  
  if (ctx.Key.StartsWith("bar/")) {
    return ctx.DuplicateDefaultEntryOptions().SetDuration(...);
  }
  
  return null;
};

Of course the user can also reference globally defined entry options to not allocate them every time, and save resources.
For example the example 3 can be:

// SOMEWHERE ELSE
public static class MyStuff {
  public static FusionCacheEntryOptions FooEntryOptions {get; set; }
  public static FusionCacheEntryOptions BarEntryOptions {get; set; }
}

// ...

// 3. MOST COMPLEX:
// - IF KEY STARTS WITH "foo/" -> CREATE A NEW ONE
// - IF KEY STARTS WITH "bar/" -> DUPLICATE THE DEFAULT ONE AND CHANGE SOMETHING
// - OTHERWISE JUST RETURN NULL (USES THE DEFAULT ONE)
DefaultEntryOptionsProvider = ctx => {
  if (ctx.Key.StartsWith("foo/")) {
    return MyStuff.FooEntryOptions ??= new FusionCacheEntryOptions().SetFailSafe(...);
  }
  
  if (ctx.Key.StartsWith("bar/")) {
    return MyStuff.BarEntryOptions ??= ctx.DuplicateDefaultEntryOptions().SetDuration(...);
  }
  
  return null;
};

This should basically cover any possible need (hopefully).

Thoughts?

[jodydonetti]

Copy pasting from the OG issue:

Hi @gleb-osokin-gen , I'm exploring the implementation of the feature, looking at edge cases and whatnot.

Something that I'd like to know your view on is how to define this new thing.
I see 2 possibilities to define the "entry options builder", as either one of these:

"is used to get a per-key DefaultEntryOptions"

or:

"is used to get a per-key entry options when nothing is passed to the main method call (eg: GetOrSet)"

The difference is subtle, and it is mainly about what should happen when something like this is called:

cache.GetOrSet(
  "foo",
  factory,
  options => options.SetDuration(...)
);

Right now FusionCache takes the DefaultEntryOptions, duplicates it, and apply the logic specified with options => options....

So the question is: with this new "entry options builder" thing should it:

"start from the normal DefaultEntryOptions"

or:

"start from the per-key one, if the logic has been specified"

?

I have a clear answer in my mind for... reasons, but I'd like to see your point of view.

Thanks!

[gleb-osokin]

Hey @jodydonetti , thank you so much, the design looks very promising and it will indeed enable using custom configurations from, e.g., IOptions, if needed 👍

As to your question, I feel the "start from the per-key one, if the logic has been specified" might be a better candidate for a few reasons:

• you can always get the DefaultEntryOptions from the cache single instance, if needed, but getting per-key options would be problematic, unless you already have them available.
• such behavior is arguably more predictable - if you have entry options provider, then it is applied everywhere uniformly.

The only drawback I see (apart from the need to adjust tons of documentation) is that developers will need to be careful when modifying the options in the GetOrSet(...) method, especially if they opt to use shared instance (like in the MyStuff.FooEntryOptions example). But, I guess, if they do apply such a micro-optimization, then they will need to apply extra care on the call site as well... Hope that makes sense.

[jodydonetti]

As to your question, I feel the "start from the per-key one, if the logic has been specified" might be a better candidate for a few reasons:

• you can always get the DefaultEntryOptions from the cache single instance, if needed, but getting per-key options would be problematic, unless you already have them available.
• such behavior is arguably more predictable - if you have entry options provider, then it is applied everywhere uniformly.

Totally agree, that was my conclusion, too.

The only drawback I see (apart from the need to adjust tons of documentation)

Eh, I know 🥲

is that developers will need to be careful when modifying the options in the GetOrSet(...) method, especially if they opt to use shared instance (like in the MyStuff.FooEntryOptions example). But, I guess, if they do apply such a micro-optimization, then they will need to apply extra care on the call site as well... Hope that makes sense.

Yes I'm currently exploring this very tricky scenario, with all the potential ramifications. It's quite complex all things considered, but I'm giving it a shot.

Will update.

[jodydonetti]

Hi @gleb-osokin , I have an update.

While working on this I come to realize that the above design is not enough, let me explain.

Currently in FusionCache I have access to 2 things:

• DefaultEntryOptions: a reference to one FusionCacheEntryOptions instance
• CreateEntryOptions(...): a method that creates a new FusionCacheEntryOptions instance every time

They give me total control of what I want to do based on what I need internally.

For example, if I just need a FusionCacheEntryOptions object that I know I'll not touch (because the user called TryGet without passing an entry options object) I can just reference the DefaultEntryOptions object directly, since it will not be changed and it's safe. Instead, if I need to create a new one starting from DefaultEntryOptions because I'll need to change it (because the user called GetOrSet with the options => options overload) I can call CreateEntryOptions(...) and be, again, safe with it.

If I now open the doors to a user-provided logic to return a FusionCacheEntryOptions as a default, without knowing if the object returned has just been created or if it's a long lasting reference, I cannot be safe in how I'll use it internally, therefore creating a problem.

Before I continue: does this make sense?

So I need to change the signature of the logic (Func<...> etc) that users can implement, in 2 possible ways:

• INPUT: add an input bool (either as an arg or to the context object) that allows FusionCache to specify to the user-provided logic if it needs to change the returned entry options, so the logic itself can just return a long lived reference or must create a new instance (via new or by duplicating the defaults)
• OUTPUT: change the return type and add a bool (maybe a value tuple? I'm still experimenting) to force users to specify if the returned entry options object is ok to be modified

They are roughly similar, but I prefer the second option since it would force users to specify true or false to basically indicate "allowing mutability", which is something that they now have to do and is reasonably harder to get wrong, instead of adding an arg that they may inadvertently ignore. Also, in case of an immutable FusionCacheEntryOptions object returned by users, the need on the FusionCache side would just be to duplicate it, which is something that FusionCache can do directly without the need to ask users to do that.

I understand it may be convoluted to grasp without the full knowledge of the internals of FusionCache, but I wanted to share the journey here to maybe get some opinions or more good ideas.

[gleb-osokin]

Hey @jodydonetti , I've double-checked the current code in main for the usages of both DefaultEntryOptions and CreateEntryOptions and found the following:

1. In 100% of FusionCacheExtMethods members, where DefaultEntryOptions is used, it is used in the following form:

return MethodName<TValue>(..., cache.DefaultEntryOptions.Duplicate(duration).SetIsSafeForAdaptiveCaching(), ...);

There are 2 more occurrences of DefaultEntryOptions usage in NullFusionCache and 1 in SetupBackplane method. The latter is internal, so we can probably, leave it as it is for the time being. As to the former, the consistent behavior would be, I guess, to replace DefaultEntryOptions with the call to the new per-key provider. But maybe you have a better idea?

2. The current non-virtual CreateEntryOptions method is defined as follows:

public FusionCacheEntryOptions CreateEntryOptions(Action<FusionCacheEntryOptions>? setupAction = null, TimeSpan? duration = null)
{
	var res = _defaultEntryOptions.Duplicate(duration);
	setupAction?.Invoke(res);
	return res;
}

In 100% of FusionCacheExtMethods members, where CreateEntryOptions(...) is used, it is then used in the following form:

return MethodName<TValue>(..., cache.CreateEntryOptions(setupAction).SetIsSafeForAdaptiveCaching(), ...);

(notice, how we never pass duration parameter, although it is allowed by the implementation).

The method is also used in FusionHybridCache.CreateFusionEntryOptions, but that one is private.

Considering 1 and 2, it seems that options are duplicated in all cases, so, technically, all occurrences of public calls to cache.DefaultEntryOptions.Duplicate(duration) can be replaced with cache.CreateEntryOptions(null, duration).SetIsSafeForAdaptiveCaching() - the default behavior is the same (apart from applying the factory). There is a caveat, though. If somebody were to implement their own IFusionCache, then CreateEntryOptions would differ in behavior from simple DefaultEntryOptions duplication. But you can already see, that even in the current implementation we are not entirely protected from somebody returning a single instance and then mutating it inadvertently later.

Now, the API (finally 😅). To be honest, I would not complicate it with any mutability markers/flags, because they will confuse the users, but won't bring much benefit - it is still possible to misuse them quite easily, i.e., promise the returned object to be immutable, but still mutate it somewhere else.

We already provide two 'obvious' options via the API in the current design:

1. I want to change something on the options => call context.DuplicateDefaultEntryOptions() and mutate at will
2. I don't want to change anything on the options => return null. Let the inner mechanism pick whatever it wants.

All the other 'not-so-obvious' possibilities, e.g. reuse single instance, etc. - can be applied at the risk of the implementors. The documentation should definitely bring up the dangers of mutating single instances of options, though.

Let me know if that makes sense.

[jodydonetti]

Hey @gleb-osokin

I've double-checked the current code in main for the usages of both DefaultEntryOptions and CreateEntryOptions and found the following:
...
Considering 1 and 2, it seems that options are duplicated in all cases, so, technically, all occurrences of public calls to cache.DefaultEntryOptions.Duplicate(duration) can be replaced with cache.CreateEntryOptions(null, duration).SetIsSafeForAdaptiveCaching() - the default behavior is the same (apart from applying the factory).

Eh, not really because there are also other cases where I'm referring directly to IFusionCache.DefaultEntryOptions, like in most public method implementations (eg: GetOrSet, TryGet, etc) for when no entry options are passed, with something like this:

options ??= DefaultEntryOptions;

Because of this, I tried a couple of different designs but in the end it boiled down to the need for the provider logic to:

• return a FusionCacheEntryOptions
• specify if it can be mutated

The design I'm currently using is this:

public abstract class FusionCacheEntryOptionsProvider
{
	public abstract FusionCacheEntryOptions? GetEntryOptions(FusionCacheEntryOptionsProviderContext ctx, string key, out bool canMutate);
}

So, an abstract class instead of a lambda, to allow usage of the out param and better for potential evolvability.

Basically the need to mutate or not the returned entry options is NOT known by the provider, which in turn should not allocate if it can avoid that (and set canMutate to false) or return a new instance every time if it can't do otherwise (and set canMutate to true).

On the FusionCache side there is a new method like this:

FusionCacheEntryOptions GetDefaultEntryOptions(string key, bool willMutate);

In this way FusionCache can be asked for a default entry options by-key but to do that it MUST be told if the intent is to mutate or not: based on these (key + willMutate) FusionCache is now checking if there is a provider or not, ask it for entry options and when getting it it will automatically compare the intent (willMutate) with what has been done by the provider (canMutate) and do the bare minimum thing to satisfy the intent requested, all while also handling no provider at all or a provider returning null, and do the automatic fallback to the DefaultEntryOptions, knowing what to do in terms of duplication etc.

For completeness, here are the revelant source codes in full (including relevant xml comments, which will show up in IntelliSense etc).

📄 FusionCacheEntryOptionsProvider.cs

/// <summary>
/// A provider to get <see cref="FusionCacheEntryOptions"/> based on a key.
/// <br/><br/>
/// ⚠️ <strong>IMPORTANT:</strong> in your GetEntryOptions() implementation carefully set the canMutate out param to indicate if the returned object can be mutated or not.
/// </summary>
public abstract class FusionCacheEntryOptionsProvider
{
	/// <summary>
	/// Provide entry options based on a key, by either returning a new instance or a reference to an existing one (for improved performance).
	/// <br/><br/>
	/// ⚠️ <strong>IMPORTANT:</strong> carefully set the <paramref name="canMutate"/> out param to indicate if the returned object can be mutated or not.
	/// </summary>
	/// <param name="ctx">The context, containing supporting features.</param>
	/// <param name="key">The cache key.</param>
	/// <param name="canMutate">An out parameter that indicate if the returned object can be mutated.</param>
	/// <returns>The entry options.</returns>
	public abstract FusionCacheEntryOptions? GetEntryOptions(FusionCacheEntryOptionsProviderContext ctx, string key, out bool canMutate);
}

📄 FusionCache.cs

// ...

public FusionCacheEntryOptions GetDefaultEntryOptions(string key, bool willMutate)
{
	FusionCacheEntryOptions? _options = null;
	bool canMutate = false;

	// TRY WITH THE PROVIDER (IF ANY)
	if (_defaultEntryOptionsProvider is not null)
	{
		_options = _defaultEntryOptionsProvider.GetEntryOptions(_defaultEntryOptionsProviderContext, key, out canMutate);
	}

	// IF NO PROVIDER OR NO OPTIONS RETURNED -> FALLBACK TO DEFAULTS
	if (_options is null)
	{
		_options = _defaultEntryOptions;
		canMutate = false;
	}

	// IF CAN MUTATE -> WE ARE GOOD ANYWAY
	if (canMutate)
		return _options;

	// CANNOT MUTATE
	return willMutate
		? _options.Duplicate()
		: _options;
}

// ...

📄 FusionCacheEntryOptionsProviderContext.cs

/// <summary>
/// The context passed to the <see cref="FusionCacheOptions.DefaultEntryOptionsProvider"/> to get a default <see cref="FusionCacheEntryOptions"/> based on the cache key.
/// </summary>
public class FusionCacheEntryOptionsProviderContext
{
	private readonly IFusionCache _cache;

	/// <summary>
	/// Creates a new <see cref="FusionCacheEntryOptionsProviderContext"/> instance.
	/// </summary>
	/// <param name="cache">A reference to the related <see cref="IFusionCache"/> instance.</param>
	internal FusionCacheEntryOptionsProviderContext(IFusionCache cache)
	{
		_cache = cache;
	}

	/// <summary>
	/// Duplicates the <see cref="FusionCacheOptions.DefaultEntryOptions"/> to allow subsequent customizations.
	/// </summary>
	/// <returns>A <see cref="FusionCacheEntryOptions"/> instante.</returns>
	public FusionCacheEntryOptions DuplicateDefaultEntryOptions()
	{
		return _cache.DefaultEntryOptions.Duplicate();
	}
}

📄 FusionCacheOptions.cs

// ...

/// <summary>
/// Specifies a custom logic to provide a default <see cref="FusionCacheEntryOptions"/> based on the cache key.
/// <br/>
/// If not set, or if it returns <see langword="null"/>, the <see cref="DefaultEntryOptions"/> will be used instead."/>
/// <br/><br/>
/// <strong>DOCS:</strong> <see href="https://github.com/ZiggyCreatures/FusionCache/blob/main/docs/Options.md"/>
/// </summary>
public FusionCacheEntryOptionsProvider? DefaultEntryOptionsProvider { get; set; }

// ...

Thoughts?

[jodydonetti]

ps: of course I already added specific tests for this new feature + re-run the entire test suite and all looks good.

[gleb-osokin]

Oh, I think I understand now, thanks a lot for a thorough explanation! So, the problem is that we don't know in advance, whether the options, provided via FusionCache.GetDefaultEntryOptions(...) will be mutated by the user or not.

I'm just thinking: would it be possible for us to only expose the mutation-safe (i.e., duplicating) method to the public, while keeping the actual factory (potentially non-mutating method) internal? That is, we will always duplicate the options, when used for a GetOrSet overload with a setup action and keep the original otherwise:

internal FusionCacheEntryOptions GetOrCreateDefaultEntryOptions(string key)
{
    return _defaultEntryOptionsProvider?.GetEntryOptions(_defaultEntryOptionsProviderContext, key) ?? 
           _defaultEntryOptions;
}

public FusionCacheEntryOptions CreateDefaultEntryOptions(string key)
{
	return GetOrCreateDefaultEntryOptions(key).Duplicate();
}

Since the implementor is providing some custom options provider, they can probably use it from outside as well, so we ourselves don't need to expose the potentially non-mutating factory method.

To be honest, I'm extremely reluctant to provide an additional bool flag (or expose two different methods, for that matter), because it might become quite brittle that way - one might implement a call with willMutate == false, but then change something later on and break things...

There is probably something else that I'm missing, so please let me know if it makes sense.

[gleb-osokin-gen]

Also, since CreateDefaultEntryOptions doesn't know, whether GetOrCreateDefaultEntryOptions has created a mutable options, we might need to have an out parameter: GetOrCreateDefaultEntryOptions(string key, out bool isNew) as you suggested, of course.