Skip to content

Commit 396745f

Browse files
blackboxswblackboxsw
committed
chore!: switch to meson.build due to PEP517 setuptools data_files depr
setuptools deprecated use of data_files for non-package external data files or system files for some time[1],[2] as part of PEP517[3] work. This PEP517 incompatiblity prompts some distributions to drop support for cloud-init in their operating system[4]. Rather than pursue the use of setup.py via setuptools further, shift to meson which better supports downstream operating system-level packaging of ancillary distribution of man pages, config files, udev, openrc or systemd scripts which are intended to live outside of a pure python distribution. Since cloudinit package is primarily intended to be delivered as system library and utility for initial system setup during first boot, it is out of the scope of cloud-init to support local wheel or bdist installations via pypi in custom locations outside of system packages as we want only one system-package installation of cloud-init in any environment. As such, cloud-init's use of python setuptools is not necessary. A move to meson gives a bit more flexibility to handle parameterized delivery of the supplemental config, init scripts, man pages an tooling which can be extended for cross-platform build support. This commit creates an initial draft of the work to migrate cloud-init backend to use meson >= 0.63.0. - Drop setup.py and replace that functionality with meson.build files which perform the same setup, template rendering and installs previously done by setuptools. - Add test targets and dependency install targets to meson.build - Set pyproject.toml to use meson build backend - Drop now unused setup_utils and moving pep400 func to test_tools.py - Add make clean target removal of meson buildir What is missing for this commit before merging: - cloud-init documentation page in doc/rtd explaining how to leverage meson for - a commit addressing packaging/ templates for debian, redhat and suse to represent applicable changes for RPMs/DEBs - a separate downstream example pull request for ubuntu enabling meson deb package builds References: - [1] https://setuptools.pypa.io/en/latest/references/keywords.html - [2] https://setuptools.pypa.io/en/latest/userguide/datafiles.html#non-package-data-files - [3] https://peps.python.org/pep-0517/ Fixes: GH-5027
1 parent 6f8d157 commit 396745f

File tree

22 files changed

+332
-427
lines changed

22 files changed

+332
-427
lines changed

‎config/cloud.cfg.d/meson.build‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#https://mesonbuild.com/FAQ.html#but-i-really-want-to-use-wildcards
2+
c = run_command(find, '-name', '*.cfg', check: true)
3+
install_data(
4+
c.stdout().strip().split('\n'),
5+
install_dir: join_paths(sysconfdir, 'cloud', 'cloud.cfg.d'),
6+
install_tag: 'config'
7+
)
8+
install_data(
9+
['README'],
10+
install_dir: join_paths(sysconfdir, 'cloud', 'cloud.cfg.d'),
11+
install_tag: 'config'
12+
)

‎doc/examples/meson.build‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#https://mesonbuild.com/FAQ.html#but-i-really-want-to-use-wildcards
2+
c = run_command(find, '-name', '*.txt', '-o', '-name', '*.yaml', check: true)
3+
install_data(
4+
c.stdout().strip().split('\n'),
5+
install_dir: join_paths(
6+
get_option('datadir'), 'doc', 'cloud-init', 'examples'
7+
),
8+
install_tag: 'docs'
9+
)

‎doc/examples/seed/meson.build‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#https://mesonbuild.com/FAQ.html#but-i-really-want-to-use-wildcards
2+
install_data(
3+
['README', 'meta-data', 'user-data'],
4+
install_dir: join_paths(
5+
get_option('datadir'), 'doc', 'cloud-init', 'examples', 'seed'
6+
),
7+
install_tag: 'docs'
8+
)

‎doc/rtd/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@ projects, contributions, suggestions, fixes and constructive feedback.
100100
Contribute to code <development/contribute_code.rst>
101101
Contribute to docs <development/contribute_docs.rst>
102102
Community <development/summit.rst>
103+
Packaging cloud-init <development/packaging.rst>
103104

104105

105106
.. LINKS

‎meson.build‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
project(
2+
'cloud-init',
3+
version: run_command('./tools/read-version', check: true).stdout().strip(),
4+
meson_version: '>=0.63.0',
5+
default_options: [
6+
# The default can yield broken results.
7+
'python.install_env=auto'
8+
]
9+
)
10+
completions = dependency('bash-completion')
11+
systemd = dependency('systemd')
12+
system = host_machine.system()
13+
14+
libdir = get_option('libdir')
15+
pkgdatadir = get_option('datadir')
16+
sysconfdir = get_option('sysconfdir')
17+
INIT_SYSTEM = get_option('init_system')
18+
VALID_INIT_SYSTEMS = [
19+
'systemd', 'sysvinit', 'sysvinit_deb', 'sysvinit_dragonfly',
20+
'sysvinit_freebsd', 'sysvinit_netbsd', 'sysvinit_openbsd'
21+
]
22+
if not VALID_INIT_SYSTEMS.contains(INIT_SYSTEM)
23+
error(
24+
'Invalid init_system supplied: "@0@". Must be one of: @1@'.format(
25+
INIT_SYSTEM, VALID_INIT_SYSTEMS)
26+
)
27+
endif
28+
29+
30+
DISTRO = get_option('distro')
31+
BSD_DISTROS = ['dragonfly', 'freebsd', 'netbsd', 'openbsd']
32+
RHEL_DISTROS = [
33+
'almalinux', 'centos', 'cloudlinux', 'eurolinux', 'fedora', 'miraclelinux',
34+
'rhel', 'rocky', 'virtuozzo'
35+
]
36+
VALID_DISTROS = [
37+
'alpine', 'amazon', 'debian', 'mariner', 'opencloudos', 'openeuler',
38+
'photon', 'ubuntu', 'suse', ''
39+
] + RHEL_DISTROS + BSD_DISTROS
40+
41+
if not VALID_DISTROS.contains(DISTRO)
42+
error(
43+
'Invalid distro supplied: "@0@". Must be one of: @1@'.format(
44+
DISTRO, VALID_DISTROS)
45+
)
46+
endif
47+
48+
49+
if BSD_DISTROS.contains(system) or BSD_DISTROS.contains(DISTRO)
50+
if INIT_SYSTEM = ''
51+
if DISTRO == ''
52+
DISTRO = system
53+
endif
54+
INIT_SYSTEM = 'sysvinit_' + DISTRO
55+
endif
56+
else
57+
if INIT_SYSTEM == ''
58+
INIT_SYSTEM = 'systemd'
59+
endif
60+
if DISTRO == ''
61+
fs = import('fs')
62+
if fs.is_file('/etc/redhat-release')
63+
message('Setting distro to rhel because of host /etc/redhat-release')
64+
DISTRO = 'rhel'
65+
endif
66+
endif
67+
endif
68+
69+
if RHEL_DISTROS.contains(DISTRO)
70+
lib_exec_dir = join_paths('/usr', get_option('libexecdir'), 'cloud-init')
71+
else
72+
lib_exec_dir = join_paths('/usr', 'lib', 'cloud-init')
73+
endif
74+
message('LIBEXEC: @0@ @1@'.format(lib_exec_dir, DISTRO))
75+
76+
77+
pymod = import('python')
78+
python = pymod.find_installation('python3', required: true)
79+
python.dependency()
80+
81+
find = find_program('find')
82+
83+
# TODO(When RHEL support meson >= 1.0.0 use install_sources 'preserve_path: true')
84+
py_sources = run_command(find, 'cloudinit', '-maxdepth', '1', '-name', '*.py', check: true).stdout().strip().split('\n')
85+
py_sources_files = files(py_sources)
86+
python.install_sources(py_sources, subdir: 'cloudinit')
87+
88+
py_source_dirs = run_command(find, 'cloudinit', '-type', 'd', '-not', '-name', '*pycache*', check: true).stdout().strip().split('\n')
89+
foreach source_subdir: py_source_dirs
90+
py_sources = run_command(find, source_subdir, '-maxdepth', '1', '-name', '*.py', check: true).stdout().strip().split('\n')
91+
python.install_sources(py_sources, subdir: source_subdir)
92+
endforeach
93+
94+
bash_completions_dir = completions.get_variable(
95+
pkgconfig: 'completionsdir', default_value: '/etc/bash_completion.d')
96+
install_data(
97+
'bash_completion/cloud-init', install_dir: bash_completions_dir, install_tag: 'scripts')
98+
99+
foreach script: ['tools/ds-identify', 'tools/hook-hotplug','tools/uncloud-init', 'tools/write-ssh-key-fingerprints' ]
100+
install_data(
101+
script, install_dir: lib_exec_dir, install_tag: 'scripts')
102+
endforeach
103+
104+
105+
# Scripts
106+
foreach script: files(
107+
['tools/cloud-init-per', 'tools/cloud-id', 'tools/cloud-init'])
108+
install_data(
109+
script,
110+
install_mode: 'rwxr-xr-x',
111+
install_dir: get_option('bindir'),
112+
)
113+
endforeach
114+
115+
# Schemas
116+
117+
# Required Config and Templates
118+
render_tmpl = './tools/render-template'
119+
subdir('config/cloud.cfg.d')
120+
subdir('templates')
121+
122+
if INIT_SYSTEM == 'systemd'
123+
udev = dependency('udev')
124+
systemd_unit_dir = systemd.get_variable(
125+
pkgconfig: 'systemdsystemunitdir'
126+
)
127+
systemd_generator_dir = systemd.get_variable(
128+
pkgconfig: 'systemdsystemgeneratordir'
129+
)
130+
subdir('systemd')
131+
udev_dir = udev.get_variable(
132+
pkgconfig: 'udevdir', default_value: '/usr/lib/udev'
133+
)
134+
install_data(
135+
'udev/66-azure-ephemeral.rules',
136+
install_dir: join_paths(udev_dir, 'rules.d'),
137+
install_tag: 'systemd'
138+
)
139+
140+
# Must generate systemd templates in root mesonbuild, because nested
141+
# systemd/meson.build results in builddir @OUTPUT@ macro being doubly-nested
142+
# paths
143+
systemd_generators = run_command(find, 'systemd', '-name', '*generator*.tmpl', check: true).stdout().strip().split('\n')
144+
foreach template: systemd_generators
145+
custom_target(
146+
input: template,
147+
output: '@BASENAME@',
148+
command: [render_tmpl, '@INPUT@', join_paths(meson.current_build_dir(), '@OUTPUT@')],
149+
install: true,
150+
install_dir: systemd_generator_dir,
151+
install_tag: 'systemd'
152+
)
153+
endforeach
154+
155+
systemd_templates = run_command(find, 'systemd', '-not', '-name', '*generator*', '-name', '*.tmpl', check: true).stdout().strip().split('\n')
156+
foreach template: systemd_templates
157+
custom_target(
158+
input: template,
159+
output: '@BASENAME@',
160+
command: [render_tmpl, '@INPUT@', join_paths(meson.current_build_dir(), '@OUTPUT@')],
161+
install: true,
162+
install_dir: systemd_unit_dir,
163+
install_tag: 'systemd'
164+
)
165+
endforeach
166+
endif
167+
168+
# Schema files
169+
json_schemas = run_command(find, 'cloudinit', '-name', '*.json', check: true).stdout().strip().split('\n')
170+
python.install_sources(json_schemas, subdir: join_paths('cloudinit', 'config', 'schemas'))
171+
172+
custom_target(
173+
input: 'config/cloud.cfg.tmpl',
174+
output: 'cloud.cfg',
175+
command: [render_tmpl, '--is-yaml', '@INPUT@', join_paths(meson.current_build_dir(), '@OUTPUT@')],
176+
install: true,
177+
install_dir: join_paths(sysconfdir, 'cloud'),
178+
install_tag: 'config'
179+
)
180+
181+
182+
# Docs and Examples:
183+
subdir('doc/examples')
184+
subdir('doc/examples/seed')
185+
mandir = get_option('mandir')
186+
foreach man_page : ['cloud-init.1', 'cloud-id.1', 'cloud-init-per.1']
187+
install_data(
188+
'doc/man/' + man_page,
189+
install_dir: join_paths(mandir, 'man1'),
190+
install_tag: 'docs'
191+
)
192+
endforeach
193+
194+
# Prerequisite for using install_deps or install_test_deps:
195+
# python3 -m venv ../.venv
196+
# source ../.venv/bin/activate
197+
# meson setup ../builddir -Dinit_system=systemd -Ddistro=ubuntu
198+
# meson compile -C ../builddir install_deps
199+
# meson compile -C ../builddir install_test_deps
200+
run_target(
201+
'install_deps',
202+
command: [python, '-m', 'pip', 'install', '-r', meson.project_source_root() / 'requirements.txt'],
203+
)
204+
run_target(
205+
'install_test_deps',
206+
command: [python, '-m', 'pip', 'install', '-r', meson.project_source_root() / 'test-requirements.txt'],
207+
)
208+
209+
# CLI: meson test -C ../builddir py3 -v
210+
test(
211+
'py3',
212+
python,
213+
args: ['-m', 'pytest', meson.project_source_root() / 'tests' / 'unittests'],
214+
timeout: 180,
215+
is_parallel: true
216+
)

‎meson_options.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
option('init_system', type: 'string', value: 'systemd', description: 'Set target init system. Choices: systemd, sysvinit, sysvinit_deb, sysvinit_freebsd, sysvinit_netbsd, sysvinit_openrc. Default: systemd.')
2+
option('distro', type: 'string', value: '', description: 'Set target distribution. Choices: alpine, centos, debian, dragonfly, fedora, freebsd, netbsd, openbsd, photon, redhat, rocky, ubuntu. Default: "". When not set, determine distro from cloudinit.util.get_distro_info')

‎packages/bddeb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ def find_root():
1919
top_dir = os.path.dirname(
2020
os.path.dirname(os.path.abspath(sys.argv[0]))
2121
)
22-
if os.path.isfile(os.path.join(top_dir, "setup.py")):
22+
if os.path.isfile(os.path.join(top_dir, "meson.build")):
2323
return os.path.abspath(top_dir)
2424
raise OSError(
2525
(

‎packages/brpm‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ def find_root():
1616
top_dir = os.path.dirname(
1717
os.path.dirname(os.path.abspath(sys.argv[0]))
1818
)
19-
if os.path.isfile(os.path.join(top_dir, "setup.py")):
19+
if os.path.isfile(os.path.join(top_dir, "meson.build")):
2020
return os.path.abspath(top_dir)
2121
raise OSError(
2222
(

‎packages/debian/rules‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ INIT_SYSTEM ?= systemd
66
export PYBUILD_INSTALL_ARGS=--init-system=$(INIT_SYSTEM)
77

88
%:
9-
dh $@ --with python3 --buildsystem pybuild
9+
dh $@ --buildsystem meson
1010

1111
override_dh_auto_test:
1212
ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))

‎packages/pkg-deps.json‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
{
22
"debian" : {
33
"build-requires" : [
4+
"meson",
5+
"bash-completion",
6+
"systemd-dev",
47
"debhelper",
58
"dh-python",
69
"python3-debconf"
@@ -43,7 +46,10 @@
4346
},
4447
"redhat" : {
4548
"build-requires" : [
46-
"python3-devel"
49+
"meson",
50+
"ninja-build",
51+
"python3-devel",
52+
"systemd-devel"
4753
],
4854
"requires" : [
4955
"e2fsprogs",

0 commit comments

Comments
 (0)