Layer Crossing Definitions Package Versioning and Architecture

[anthony-murphy]

Currently within the fluid framework we have several definition packages that we version and release independently. These packages are:

• @fluidframework/common-definitions
• @fluidframework/protocol-definitions
• @fluidframework/core-interfaces
• @fluidframework/driver-definitions
• @fluidframework/container-definitions

These packages have a specific goal which is to provide a stable api for any layer crossing object which is agnostic to specific implementation versions. The specific layers of concern here are server, loader, and runtime. These layers are special as the fluid framework explicitly supports independent compilation and packaging of each layer with no shared implementations. The definition packages specify the contracts across these layers and allow objects to move across these layers at runtime.

The versions of these packages have been pinned as an interim solution to meeting the above goal. However, pinning the versions is not sustainable long term. This issue will lay out several proposals which should allow us to meet our goal for these packages while enabling forward progress of the definitions packages and their respective implementations.

Remove non-layer crossing contracts

Within the current definitions packages there are some non-layer crossing types. These types unnecessarily bloat our definitions packages and limit our agility in modifying the implementation tied to them. In general, these types should be moved to the packages containing their respective implementations and/or new packages within those layers. Additionally, there may be some types that are unused, and can be fully removed.

Export Definitions as Types only

Typescript support both package imports and exports as types only. This means that there will be no runtime impact from those import or exports. We should update our definition packages to export everything as type only. By exporting as types only we can truly leverage duck typing for back compat, without needed to consider runtime compatibility. Runtime compatibility is a problem we will solve at the implementation layer. Practically this means the removal of all Enum and functions from our definition packages

Re-export upstream definitions

This is more a nice to have, but i think it provides a number of benefits.

• Make it easier for consumers to manage definition dependencies, as they should only need a single import per layer
• Gives us some flexibility in moving types between layers in a way that minimizes breaks for downstream consumers.
• Makes peerDependencies more supportable

Validation

We currently have tests that validate our code is back compat at runtime, however we don't do any compile time validation. Some ideas here are leverage peer dependencies, and validate multiple compiles, likely old version code with new dependencies. another potential option is to leverage some of the type testing we do to ensure our new types are back compat to some point in the past.

[anthony-murphy]

I think we'll need to solve #3264, and #2566 before we can do much here, as the current Module augmentation used by IFluidObject doesn't work correctly with re-exporting, and in general will create a lot of cross version pain. The biggest unknown here is how to deal with types that leverage keyof IFluidObject. I don't think there is a one size fits all approach to replacing that, but i'm exploring the usage patterns to see what we can do.