The documented `.git_archival.txt` causes git archives to change hash after first post-release commit is made

[mgorny]

The documentation suggests using the following content for the .git_archival.txt file:

node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
ref-names: $Format:%D$

This means that if a tag is created on top of the main branch, the file initially contains e.g.:

ref-names: HEAD -> main, tag: v1

but once another commit is added and main no longer corresponds to the tag, it changes to:

ref-names: tag: v1

This causes the git archive hash to change. This is a problem for distributions like Gentoo that are using git archives generated by GitHub. The initial archive that we fetch once the release is made changes once upstream makes new commits.

See e.g.: https://bugs.gentoo.org/895910, https://bugs.gentoo.org/895712

[RonnyPfannschmidt]

Thanks for the heads up

Is there a suggested way to get that data

[mgorny]

I don't really know. I didn't even know such a thing is possible before setuptools_scm implemented it ;-).

[RonnyPfannschmidt]

Based on the git docs it's impossible to opt out of the problem details

Is there any reason why you don't use the release artifacts from pypi?

[ionenwks]

Is there any reason why you don't use the release artifacts from pypi?

In a lot of cases it's because they're missing something, e.g. tests (Edit: otherwise Gentoo does prefer pypi tarballs). Ideally everyone would include them but it's not always so easy.

[RonnyPfannschmidt]

That's unfortunate, in particular since setuptools_scm typically enforces that all files go in

[mgorny]

Yes, it is. It's been last debated in https://discuss.python.org/t/should-sdists-include-docs-and-tests/14578, with no definite conclusion. Unfortunately, some people believe that "sdist is only for pip" and so including files that pip (or a PEP517 build system, more generally) doesn't use is a waste of space.

[webknjaz]

Unfortunately, some people believe that "sdist is only for pip" and so including files that pip (or a PEP517 build system, more generally) doesn't use is a waste of space.

@mgorny I've been pointing people at my action https://github.com/re-actors/checkout-python-sdist to try to bring better standard into the ecosystem FYI. Projects that would use this, would probably be more downstream-friendly, as a result.

[eli-schwartz]

Isn't the need for ref-names obsoleted by the availability of the describe-name I added to current versions of git (and which is specifically designed to be stable, at least if abbrev is used)?

Maybe the template recommendation can simply phase out the old and unreliable ref-names formatter?

[maresb]

I have the impression that describe-name is relatively new and not so widespread yet. Specifically, I've noticed often warnings about describe-name being unavailable. When I locally build the aesara package under Ubuntu 20.04 (git 2.39.2), I get

* Building wheel...
/tmp/build-env-u3aklikk/lib/python3.11/site-packages/setuptools_scm/git.py:295: UserWarning: git
archive did not support describe output

Thus I fear that in practice removing ref-names could break a lot of workflows.

[eli-schwartz]

When locally building from a git repository the primary data source should be git itself, not the .git_archival.txt file... I wonder why this is triggering a warning.

[maresb]

Yes, that's a very good question, and I'd like to look into that. But in any case, I'm still concerned about the prevalence of Git versions which don't support describe-name.

[eli-schwartz]

In theory it should only matter in cases where you are not building from a git checkout, and you're not building from a PyPI sdist, and you are only building from the output of running git archive.

The main situation where that happens is on servers such as github.com, gitlab.com, git.sr.ht, codeberg.org, etc. and those tend to roll out new git updates much more often than new Ubuntu LTSes.

It is a good question, how often that will cause issues for local workflows. But I'm hoping the answer might be "not very".