Should file_format follow semantic versioning?

[jack-berg]

file_format currently includes major and minor, and but no patch version. This was originally discussed here.

Should we revisit this and include patch as well?

[tsloughter]

Patch may always be 0 but is there anything gained from leaving out the .0? So I'd lean towards including it just so its not "different".

[MikeGoldsmith]

+1 - even if we never use it, it's good to have the option.

[pirgeo]

Is there any downside to having it? On the other hand: Would there ever be a situation where we want to release a patch version?

[MikeGoldsmith]

I don't think there are any negatives to having it, beyond it never getting used.

[marcalff]

Note that nothing requires the file format to be a version number currently, this is what the examples do, but not what the schema enforces:

"properties": {
        "file_format": {
            "type": "string"
        }
    },

This opens two underlying questions to address.

Requirements on the parser implementation

We need to define what an implementation is expected to do, when parsing a config.yaml document.

Assume an implementation supports a format known as "1.10.20".

When a document that contains the following formats is found, we need to clarify what the reasonable behavior is for the implementation of config.yaml:

For example

• format: "1.10.20"
  • accept the document and proceed, the parser knows how to read it.
• format "2.x.y"
  • reject, unsupported format.
• format: "1.10.25"
  • unsure, needs to be clarified
• format: "1.10.15"
  • unsure, needs to be clarified
• format: "1.12.x"
  • unsure, needs to be clarified
• format: "1.8.x"
  • unsure, needs to be clarified

Guidelines for schema versioning

When the schema itself changes, another question is when to change the major / minor / patch version number.

For example:

• add an optional attribute
  • increase the patch version ?
• add a mandatory attribute
  • increase the minor version ?

makes sense, but the the story gets more complicated very quickly.

When adding a mandatory attribute to a node that itself is optional (say, define configuration for a new exporter), is this a change in patch version only, or a change in minor version ?

These guidelines needs to be clarified before reaching schema 1.0.0

[jack-berg]

Great summary @marcalff.

Some thoughts:

• Implementations should only need to support 1 major version of the schema. Its feasible to support multiple, but I think doing so essentially requires two copies of the code, so its a lot to ask maintainers. For this reason, we (config working group) should try hard to avoid breaking changes.
• If the file format specified by the user is aligned with the major version, then we should be parse / interpret the contents, but the resulting SDK components may not be what the user wanted. While not ideal, I think this is better than a strict approach where the implementation fails if version isn't aligned. But still, the user needs to be made aware that discrepancies may exist. To accommodate this, I think we should mandate that implementations log a warning when the implemented major / minor version is lower than the user specified major / minor. I don't think such a warning is necessary if the implemented major / minor version is greater than the user specified major / minor, and think we can define versioning rules that ensure that the user is always getting what they expect in this case.
• I think we should reserve patches for bugs in the schema. Minor versions can have new additional options properties. Major versions for new required properties, breaking changes in the semantics, etc.
• I think we should consider only including the major / minor in the user specified file_format property. Behind the scenes, the implementation needs to target a release with a major / minor / patch, but if the patch only contains bug fixes, the user doesn't need to be concerned with specifying it. Ideally, implementations stay up to date with the latest patch for whatever minor version they target. (And ideally, we only publish patches very soon after the minor releases they fix to limit that chance of implementations targeting a buggy release.) Maybe we specify that its not an error to include the patch, but its also not required.

When adding a mandatory attribute to a node that itself is optional (say, define configuration for a new exporter), is this a change in patch version only, or a change in minor version ?

By my logic, this is a breaking change that required a major version. Therefore, its extremely important to get the mandatory fields correct the first time we release any new type. In my experience, this is common in schema versioning.

[jack-berg]

Solved in #142.