New Release?

[aarondvail]

Checklist

• I'm asking a question
• I've looked through the README and FAQ for similar questions
• I've searched the bugtracker for similar questions including closed ones

Question

Preface:
There have been a number of fixes done since the last release (2021-12-17) {over a years worth} The error messages refer using youtube-dl -U (which is already in place). All the errors I am experiencing are already closed issues, but no no release so youtube-dl -U won't fix it. Issue #31583

Question:
Can either the instructions be updated (and the error message refering to youtube-dl -U be dropped) to "install" via a github pull from Master

or

Can a new release be cut?

Thanks

[dirkf]

A new release is overdue but depends on solving certain cross-platform issues in the CI build process. Meanwhile for the main platforms, see #31530 for update hints.

[Grub4K]

What are those issues that need sorting out? Is there an issue/PR tracking this? Creating a PR could bring in people that are willing to help with the process (like myself)

[dirkf]

#30644, but it should probably be refreshed now.

This is very far down my stack today, but as I recall there were various possible blockers:

• change to pytest for Py3.10+ (mostly achieved in my test branch with a pytest build that should run on the supported platforms)
• especially Jython was tricky, if anyone cares (but we have found a Py2.6 user today)
• various workflow upgrades because of GH dependencies
• the usual maze of twisty build system dependencies, all alike.

Obviously I need to dive back into this especially in view of yt-dlp/yt-dlp#6220, which I would also like to acquire, if only for more regular versioned releases.

[dirkf]

But isn't the auto-update installer already there, depending on the how the yt-dl instance was installed? Once a build has been deployed to the release page and to PyPi, it can be pulled by almost all installation types.

The problem being addressed is really to get the small number of builds in the release tested on all the different platforms.

My understanding is that PyPi and other options would only update on release, and not necessarily auto update at that (i.e. without running pip install again separately).

My suggestion would auto update on either release or master branch updates before each run of the program. That way you don't need to rush to cut release, at the same time users could stay up to date if they wish.

One thing I didn't really point out is that youtube_dl itself would also be initiated through this script, so it would run the checks to update and then pass the argument to youtube_dl and run.

Basically it's a simple hack to make updating against the master branch painless and easy to the user.

[Grub4K]

Will #30644 keep tracking this and should I post my remarks on there?

I think maybe instead of perfecting the release process it might make sense to release something so as to get updates to the end user. If releases were not automated, whats problematic about building like that, then passing those executables to the user to be tried on various platforms? The related release can be marked as pre-release if need be, but that way there at least are executables which will work for most people while we can get reports for the edge cases and squash those specifically.

I would also advise to look into the release.yml/release-nightly.yml workflow in yt-dlp/yt-dlp#6220. It is crafted so that it creates a release in the yt-dlp style regardless of build output, adding all of the build artifacts from build.yml, it's checksums and the _update_spec. A build.yml satisfying simplistic conditions can be created at first, which would build some more general artifacts, gathering info about possible incompatibilities.

These are the repository configuration options of the release and nighly action:

vars.PUSH_VERSION_COMMIT: Update master branch with a push
vars.BUILD_NIGHTLY: Build a nightly release on every commit

# Both need to be set to push build to archive repo as release
vars.ARCHIVE_REPO: repo like `yt-dlp/yt-dlp-builds`
secrets.ARCHIVE_REPO_TOKEN: token with contents:write

# Push to PyPI and brew if each is set
secrets.PYPI_TOKEN: Token for PyPI
secrets.BREW_TOKEN: Token for brew

update-version.py would have to be backported since the format changed and it's output is now a bit different. Maybe the naming could also be changed to update_version.py?

make_changelog.py, while configurable, is most likely not configurable enough to be able to support the ydl changelog. The author assumption would have to be corrected (Authored by: ) or the changelog would have to be crafted and read manually for now (would be simple to do a shim that just reads from ChangeLog for now).

• change to pytest for Py3.10+ (mostly achieved in my test branch with a pytest build that should run on the supported platforms)

Are the test adjustments needed for the release for now? Is that preparation to be testing if the builds would work as expected? If so, I'd think that can be moved back for now?

• especially Jython was tricky, if anyone cares (but we have found a Py2.6 user today)

Do we need to supply youtube-dl builds that embed JPython? Otherwise I can't really see the direct relation of that and a release.

• various workflow upgrades because of GH dependencies

This should be handled by 6220. I can help with the remaining blockers there.

• the usual maze of twisty build system dependencies, all alike.

Yes, understandable. If we encounter anything in particular we might need to push that back until somebody with experience on that system or with those dependencies can chime in. I think those issues will become apparent once we create/try creating the actual build.yml.

The steps that I would propose for now are:

• Create the release.yml and adjust to the format we would want
• Fix/port update_version.py and make_changelog.py
• Create a simplified build.yml handling the most common targets.
• Publish a pre-release with the created executables (people will be _{^{sort of}} happy here)
• Fix problems, improve and extend build
• Employ extended testing and the sort

Feel free to contact me via E-Mail if you would like to move to a different communication channel for this. Alternatively let me know of anything else I can do.