Almost any interface change in the Fluid repo results in IFluidObject typing incompatibility

[leeviana]

Describe the bug

Anything that references an IFluidObject type from different versions are pretty much always guaranteed to be typescript incompatible in a "major version" sort of way because of the number of "provides" contracts on IFluidObject . This means we can pretty much never have two different versions of packages on either side of an IFluidObject exchange, regardless of whether you're even using any interfaces that have actually changed. For instance, if somebody makes a random change to IFluidDataStoreContext, through a really deep dependency chain involving views and IHostRuntime, this essentially means a major bump in IFluidObject/core-interfaces.

To Reproduce

Make a change to almost any interface in the Fluid repo. Likely the IFluidObject interface in the version of the code published is not compatible with the version of code right before.

Expected behavior

IFluidObject interfaces should be somewhat durable across Framework changes, especially in interfaces that aren't being immediately used.

[leeviana]

@anthony-murphy

[markfields]

We chatted about this issue today, and I wanted to jot down some notes.

We discussed the approach of removing IFluidObject from API signatures, and instead using it as-needed to cast through to get a type safe interface implementation. This will of course fix the TypeScript compilation errors, but we're still left with the runtime impact of interface changes across versions, so not much appears to be gained here actually.

The bulletproof way to deal with this is to never change any interface. Instead of modifying IFoo, you create IFoo2 with the changes, leaving IFoo intact. This way, within the Fluid ecosystem, if you ever find yourself with an object with a property IFoo you know its shape definitively.

But this is pretty onerous, so we're hopeful to find a more elegant solution.

[markfields]

Update to my previous comment - even if we neutralize the type in API signatures, there's still a compile-time trap when a particular bit of code (e.g. a Container specification) imports two different module (e.g. 2 data stores) built/published against different versions of the runtime, with different definitions of IFluidObject.

I wonder if we can't just put a stake in the ground to say that when authoring a Container all included Data Stores and DDS's must be targeting the same version of the runtime? Is this just kicking the can until we get to dynamic loading? Does it even solve this issue?

[markfields]

Also, I think this issue would benefit from a concrete repro case and description of what specific errors come up. @leeviana if you can point to a PR or something that fixed this issue before, or any other collateral that would be great. On our side, @anthony-murphy and I discussed just crafting a toy example where some code imports two different modules, each with their own differing definitions of IFluidObject. Then we can not only see the errors but play around with different scenarios/tactics.

Speaking of tactics, a few we've discussed:

• Audit interfaces being registered with IFluidObject to shed some that may be abusing the pattern. @anthony-murphy and I did a quick pass and I opened issue Remove certain "internal" interfaces from IFluidObject #3264 for some potential low-hanging fruit here. This would hopefully reduce the churn.
• See if JavaScript's duck typing lets us be a bit loosey-goosey (all the water fowl!) in making changes to interfaces - e.g...
  • An extreme example would be to make everything on our interfaces optional via Partial<T>, and then even once you have an interface you need to query to see which members are actually there. I think this has a decent shot at working with the double import.
  • Using a discriminated union of versions of an interface, so the thing you get back only has the version field and anything common across all versions. Not sure how the double import would handle this... probably won't work.
• Incorporate some kind of "strong name" semantics where interfaces have the friendly name concatenated with a guid, so you can use the guid to definitively address a particular interface, but you still get friendly names. We've discussed a similar scheme for naming factories. And just don't ever change the interface without renaming it / regenerating a guid.

[markfields]

@curtisman, @anthony-murphy - These notes all came from discussions we had, anything else to add or clarify?

[leeviana]

I don't think we have a specific PR because this particular issues prevents us from checking in any code :) I believe I created this issue during one of Kenneth's particularly convoluted FFX bumps where I suggested that in order to reduce regressions/catch them more quickly, we check in the component/container version bump first and then the host version updates second. However, all the contracts around IComponent were broken/unable to be reused around really random things ten layers deep into each interface and ending in something very much unrelated like an interface change in containerRuntime etc.

For example, scriptor is both a host and a component. So its component dependencies needed to be updated. However it could not use the IComponent output of the Loader.resolve() because the Loader's version of IComponent was incompatible with the component's version of IComponent because of aforementioned really random changes that had nothing to do with IComponent interfaces that were used.

[leeviana]

You can probably repro this issue quite easily in the bohemia repo by picking any version bump and updating some dependencies in components/containers and not host/Loader code and seeing how it fails.