Pep632 support meson build #6326

blackboxsw · 2025-07-23T14:48:47Z

chore!: switch to meson.build due to PEP632 setuptools data_files depr

setuptools has deprecated use of data_files for external files and man pages for some time[1],[2] as part of PEP632[3] work.

This PEP632 incompatiblity prompts some distributions to drop support for cloud-init in their operating system[4].

Rather than continue to patch our dated use of setup.py via setuptools further, shift to meson which better supports downstream operating system distribution packaging of ancillary distribution of man pages, config files, udev, openrc or systemd scripts which are intended to live outside of a pure python distribution.

Since cloudinit package is primarily intended to be delivered as core library and utility for system setup during first boot of a system, it is out of the scope of cloud-init to support local wheel or bdist installations via pypi in custom local locations outside of system packages as we want only one system-package level installation of cloud-init in any environment.

As such, cloud-init's use of python setuptools is not necessary to package or deliver python via standard system packages, so a move to meson gives a bit more flexibility to handle delivery of the supplemental config, init scripts, man pages and tooling in one place.

This commit creates an initial draft of the work to move cloud-init build tooling from setuptools to meson.

It drops setup.py and replaces that functionality with a number of meson.build files to perform the same setup, template rendering and installs previously done by setuptools. It sets pyproject.toml to use meson build backend and supports deb building only at the moment.

What remains for this commit before moving forward

a separate downstream example pull request for ubuntu enabling meson deb package builds

References:

Fixes: GH-5027
Closes: GH-3986
LP: #1978328

Extra context:

paired with downstream ubuntu/devel branch #6394 once this lands

Test procedure

# Test meson build functionality locally:
 python3 -m venv ../.venv
source ../.venv/bin/activate
meson setup ../builddir -Dinit_system=systemd

# test package builds

./tools/run-container --package --source-package rockylinux/9 --keep
./tools/run-container --package --source-package fedora/42 --keep
./tools/run-container --package --source-package ubuntu-daily:plucky --keep

holmanb

First pass. Thanks for your work on this @blackboxsw!

config/cloud.cfg.d/meson.build

meson.build

packages/pkg-deps.json

meson.build

doc/rtd/development/packaging.rst

packages/debian/rules

meson.build

doc/rtd/development/packaging.rst

meson.build

doc/rtd/development/packaging.rst

meson.build

) - tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

…nical#6326)

tools/read-dependencies DISTRO_PKG_MAP values are only one of ubuntu, redhat, suse or fedora. Simplify value check when adding rpm-build package dependency.

Migrate package testing build info into existing testing page.

setuptools deprecated use of data_files for non-package external data files or system files for some time[1],[2] as part of PEP632[3] work. This PEP632 incompatiblity prompts some distributions to drop support for cloud-init in their operating system[4]. Rather than pursue the use of setup.py via setuptools further, shift to meson which better supports downstream operating system-level packaging of ancillary distribution of man pages, config files, udev, openrc or systemd scripts which are intended to live outside of a pure python distribution. Since cloudinit package is primarily intended to be delivered as system library and utility for initial system setup during first boot, it is out of the scope of cloud-init to support local wheel or bdist installations via pypi in custom locations outside of system packages as we want only one system-package installation of cloud-init in any environment. As such, cloud-init's use of python setuptools is not necessary. A move to meson gives a bit more flexibility to handle parameterized delivery of the supplemental config, init scripts, man pages an tooling which can be extended for cross-platform build support. This commit creates an initial draft of the work to migrate cloud-init backend to use meson >= 0.63.0. - Drop setup.py and replace that functionality with meson.build files which perform the same setup, template rendering and installs previously done by setuptools. - drop build-system backend from pyproject.toml to avoid automatic RTD meson-builds - Add test targets and dependency install targets to meson.build - Set pyproject.toml to use meson build backend - Drop now unused setup_utils and moving pep400 func to test_tools.py - update dev testing tools under packages to provide reference SPEC and debian/* files and allow for local development package builds References: - [1] https://setuptools.pypa.io/en/latest/references/keywords.html - [2] https://setuptools.pypa.io/en/latest/userguide/datafiles.html#non-package-data-files - [3] https://peps.python.org/pep-0632/ Closes: GH-3986 Fixes: GH-5027 LP: #1978328

- tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

tools/read-dependencies DISTRO_PKG_MAP values are only one of ubuntu, redhat, suse or fedora. Simplify value check when adding rpm-build package dependency.

Migrate package testing build info into existing testing page.

setuptools deprecated use of data_files for non-package external data files or system files for some time[1],[2] as part of PEP632[3] work. This PEP632 incompatiblity prompts some distributions to drop support for cloud-init in their operating system[4]. Rather than pursue the use of setup.py via setuptools further, shift to meson which better supports downstream operating system-level packaging of ancillary distribution of man pages, config files, udev, openrc or systemd scripts which are intended to live outside of a pure python distribution. Since cloudinit package is primarily intended to be delivered as system library and utility for initial system setup during first boot, it is out of the scope of cloud-init to support local wheel or bdist installations via pypi in custom locations outside of system packages as we want only one system-package installation of cloud-init in any environment. As such, cloud-init's use of python setuptools is not necessary. A move to meson gives a bit more flexibility to handle parameterized delivery of the supplemental config, init scripts, man pages an tooling which can be extended for cross-platform build support. This commit creates an initial draft of the work to migrate cloud-init backend to use meson >= 0.63.0. - Drop setup.py and replace that functionality with meson.build files which perform the same setup, template rendering and installs previously done by setuptools. - drop build-system backend from pyproject.toml to avoid automatic RTD meson-builds - Add test targets and dependency install targets to meson.build - Set pyproject.toml to use meson build backend - Drop now unused setup_utils and moving pep400 func to test_tools.py - update dev testing tools under packages to provide reference SPEC and debian/* files and allow for local development package builds References: - [1] https://setuptools.pypa.io/en/latest/references/keywords.html - [2] https://setuptools.pypa.io/en/latest/userguide/datafiles.html#non-package-data-files - [3] https://peps.python.org/pep-0632/ Closes: canonicalGH-3986 Fixes: canonicalGH-5027 LP: #1978328

) - tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

…nical#6326)

tools/read-dependencies DISTRO_PKG_MAP values are only one of ubuntu, redhat, suse or fedora. Simplify value check when adding rpm-build package dependency.

Migrate package testing build info into existing testing page.

setuptools deprecated use of data_files for non-package external data files or system files for some time[1],[2] as part of PEP632[3] work. This PEP632 incompatiblity prompts some distributions to drop support for cloud-init in their operating system[4]. Rather than pursue the use of setup.py via setuptools further, shift to meson which better supports downstream operating system-level packaging of ancillary distribution of man pages, config files, udev, openrc or systemd scripts which are intended to live outside of a pure python distribution. Since cloudinit package is primarily intended to be delivered as system library and utility for initial system setup during first boot, it is out of the scope of cloud-init to support local wheel or bdist installations via pypi in custom locations outside of system packages as we want only one system-package installation of cloud-init in any environment. As such, cloud-init's use of python setuptools is not necessary. A move to meson gives a bit more flexibility to handle parameterized delivery of the supplemental config, init scripts, man pages an tooling which can be extended for cross-platform build support. This commit creates an initial draft of the work to migrate cloud-init backend to use meson >= 0.63.0. - Drop setup.py and replace that functionality with meson.build files which perform the same setup, template rendering and installs previously done by setuptools. - drop build-system backend from pyproject.toml to avoid automatic RTD meson-builds - Add test targets and dependency install targets to meson.build - Set pyproject.toml to use meson build backend - Drop now unused setup_utils and moving pep400 func to test_tools.py - update dev testing tools under packages to provide reference SPEC and debian/* files and allow for local development package builds References: - [1] https://setuptools.pypa.io/en/latest/references/keywords.html - [2] https://setuptools.pypa.io/en/latest/userguide/datafiles.html#non-package-data-files - [3] https://peps.python.org/pep-0632/ Closes: canonicalGH-3986 Fixes: canonicalGH-5027 LP: #1978328

) - tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

…nical#6326)

tools/read-dependencies DISTRO_PKG_MAP values are only one of ubuntu, redhat, suse or fedora. Simplify value check when adding rpm-build package dependency.

Migrate package testing build info into existing testing page.

github-actions bot added the documentation This Pull Request changes documentation label Jul 23, 2025

blackboxsw marked this pull request as draft July 23, 2025 14:51

blackboxsw force-pushed the pep517-support-meson-build branch from 0892b62 to 396745f Compare July 24, 2025 02:26

blackboxsw mentioned this pull request Jul 24, 2025

The cloud-init service failed to start automatically under the openEuler system #6329

Open

blackboxsw added this to the cloud-init 25.2 milestone Jul 24, 2025

blackboxsw force-pushed the pep517-support-meson-build branch 3 times, most recently from f0cd2e5 to 6dc97e4 Compare July 24, 2025 22:46

blackboxsw marked this pull request as ready for review July 24, 2025 23:05

holmanb reviewed Jul 25, 2025

View reviewed changes

blackboxsw force-pushed the pep517-support-meson-build branch from 706fa63 to f61e0e0 Compare July 25, 2025 03:07

holmanb reviewed Jul 25, 2025

View reviewed changes

packages/debian/rules Outdated Show resolved Hide resolved

holmanb reviewed Jul 29, 2025

View reviewed changes

packages/debian/rules Show resolved Hide resolved

holmanb reviewed Jul 30, 2025

View reviewed changes

meson.build Outdated Show resolved Hide resolved

holmanb reviewed Jul 30, 2025

View reviewed changes

meson.build Outdated Show resolved Hide resolved

meson.build Outdated Show resolved Hide resolved

meson.build Outdated Show resolved Hide resolved

meson.build Outdated Show resolved Hide resolved

meson.build Outdated Show resolved Hide resolved

holmanb reviewed Jul 30, 2025

View reviewed changes

meson.build Outdated Show resolved Hide resolved

holmanb reviewed Jul 31, 2025

View reviewed changes

blackboxsw force-pushed the pep517-support-meson-build branch from f61e0e0 to 7c7717c Compare August 1, 2025 18:08

holmanb reviewed Aug 1, 2025

View reviewed changes

meson.build Outdated Show resolved Hide resolved

holmanb reviewed Aug 1, 2025

View reviewed changes

doc/rtd/development/packaging.rst Outdated Show resolved Hide resolved

blackboxsw force-pushed the pep517-support-meson-build branch 7 times, most recently from 6c423f5 to 2fbaac2 Compare August 1, 2025 20:40

holmanb reviewed Aug 1, 2025

View reviewed changes

blackboxsw force-pushed the pep517-support-meson-build branch 2 times, most recently from 69dc688 to 5880155 Compare August 1, 2025 21:28

blackboxsw added 5 commits August 22, 2025 21:44

chore: drop yaml validation target, covered in unittests (canonical#6326

7c3df6e

) - tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

chore: update developer specfile redhat/fedora and dependencies (cano…

449e7cd

…nical#6326)

chore: update developer specfile and build for opensuse (canonical#6326)

de78905

tools/read-dependencies DISTRO_PKG_MAP values are only one of ubuntu, redhat, suse or fedora. Simplify value check when adding rpm-build package dependency.

doc: add testing links to doc/rtd Development headings (canonical#6326)

5e6a6a8

Migrate package testing build info into existing testing page.

fix: read-dependencies syntax error on regex (canonical#6326)

4fd31ec

blackboxsw force-pushed the pep517-support-meson-build branch from 71cc5cf to 4fd31ec Compare August 23, 2025 03:45

blackboxsw merged commit 890873f into canonical:main Aug 23, 2025
22 checks passed

blackboxsw added a commit that referenced this pull request Aug 23, 2025

fix: add apt-get update before development pkg install (#6326)

a5d2809

blackboxsw added a commit that referenced this pull request Aug 23, 2025

chore: update debian developer packaging template (#6326)

ff9a69b

blackboxsw added a commit that referenced this pull request Aug 23, 2025

chore: drop yaml validation target, covered in unittests (#6326)

ad367e2

- tests/unittests/config/test_schema.py::TestModuleDocs and TestCloudConfigExamples - tests/unittests/test_merging.py

blackboxsw added a commit that referenced this pull request Aug 23, 2025

chore: update developer specfile redhat/fedora and dependencies (#6326)

232e8c1

blackboxsw added a commit that referenced this pull request Aug 23, 2025

doc: add testing links to doc/rtd Development headings (#6326)

fed4375

Migrate package testing build info into existing testing page.

DarkPhily pushed a commit to hetznercloud/cloud-init that referenced this pull request Sep 2, 2025

fix: add apt-get update before development pkg install (canonical#6326)

dbcc7f6

DarkPhily pushed a commit to hetznercloud/cloud-init that referenced this pull request Sep 2, 2025

chore: update debian developer packaging template (canonical#6326)

092b40b

DarkPhily pushed a commit to hetznercloud/cloud-init that referenced this pull request Sep 2, 2025

chore: update developer specfile redhat/fedora and dependencies (cano…

1129f02

…nical#6326)

DarkPhily pushed a commit to hetznercloud/cloud-init that referenced this pull request Sep 2, 2025

doc: add testing links to doc/rtd Development headings (canonical#6326)

a1883bc

Migrate package testing build info into existing testing page.

DarkPhily pushed a commit to hetznercloud/cloud-init that referenced this pull request Sep 2, 2025

fix: read-dependencies syntax error on regex (canonical#6326)

0a8b1a7

blackboxsw added a commit to blackboxsw/cloud-init that referenced this pull request Sep 3, 2025

fix: add apt-get update before development pkg install (canonical#6326)

12531e9

blackboxsw added a commit to blackboxsw/cloud-init that referenced this pull request Sep 3, 2025

chore: update debian developer packaging template (canonical#6326)

d57ab90

blackboxsw added a commit to blackboxsw/cloud-init that referenced this pull request Sep 3, 2025

chore: update developer specfile redhat/fedora and dependencies (cano…

adc2556

…nical#6326)

blackboxsw added a commit to blackboxsw/cloud-init that referenced this pull request Sep 3, 2025

doc: add testing links to doc/rtd Development headings (canonical#6326)

6ebf0d2

Migrate package testing build info into existing testing page.

blackboxsw added a commit to blackboxsw/cloud-init that referenced this pull request Sep 3, 2025

fix: read-dependencies syntax error on regex (canonical#6326)

ae8c4cb

Pep632 support meson build #6326

Pep632 support meson build #6326

Uh oh!

Conversation

blackboxsw commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Extra context:

Test procedure

Uh oh!

holmanb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

blackboxsw commented Jul 23, 2025 •

edited

Loading