Stabilize 'build.py'

[padix-key]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

In order to compile C-extensions, it is currently required to add a build script to the project directory and to pyproject.toml:

[tool.poetry]
build = "build.py"

This feature is described in #11, but not in the official Poetry documentation. So my question is whether this feature is stable. If this is the case, I would like to propose to include it in the official documentation.

[abn]

@padix-key this feature is, today, an undocumented and really unsupported features. However, we do plan on making support for build scripts more standard. This will be added to documentation once it is fully supported and flushed out. Changes like python-poetry/poetry-core#45 are stepping stones to this.

As an additional note we are also improve support for "build" configuration section along with this. So, this will become something like the following in newer versions.

[tool.poetry.build]
script = "build.py"

[albireox]

I'm confused about the new build system. I've modified my pyproject.py to use

[tool.poetry.build]
script = "build.py"
generate-setup-file = false

and

[build-system]
requires = ["poetry-core>=1.0.0a9"]
build-backend = "poetry.core.masonry.api"

and my build.py to

def build():

    distribution = Distribution({'name': 'extended', 'ext_modules': ext_modules})
    distribution.package_dir = 'extended'

    cmd = build_ext(distribution)
    cmd.ensure_finalized()
    cmd.run()

    # Copy built extensions back to the project
    for output in cmd.get_outputs():
        relative_extension = os.path.relpath(output, cmd.build_lib)
        shutil.copyfile(output, relative_extension)
        mode = os.stat(relative_extension).st_mode
        mode |= (mode & 0o444) >> 2
        os.chmod(relative_extension, mode)


if __name__ == '__main__':
    build()

where ext_modules is a list of distutils.core.Extension.

This works fine with poetry install (in the sense that the extension is compiled and I get the shared library) but when I run pip install . no compilation happens.

Am I missing something?

[albireox]

Looking at the pendulum example, it seems one also needs to add

include = [
    {path = "flicamera/*.so", format = "wheel"}
]

for the compiled extension to be installed. This should be documented when the feature becomes stable.

This include is not in the test at tests/masonry/builders/fixtures/extended_with_no_setup/pyproject.toml so I'm not sure how that test would pass.

[abn]

@albireox moving forward we will make it the responsibility if the build script to move the compiled files around such that they are in the right places as expected. If the built files are within an included package it will automatically be picked up. This is probably why it works in the test (did not look into it). If it is in another location, the include configuration is the way to go. We might try and include this wil some "not final" warning in the documentation for 1.1.0 so it is useful for maintainers till it stabilises.

[albireox]

Thanks for the explanation.

I would really encourage that if this change become part of 1.1.0 it gets documented, even if it's considered an experimental feature. The old build system, although undocumented worked as expected and a stable release should not break that.

[abn]

The old mechanism should still work out of the box; since the default is to generate the setup file for this release. But yes, I agree, some documentation explaining all this would be good.

[ahobsonsayers]

With the new poetry 1.1.0 release i have found in my projects that my line: script = "build.py" no longer works and an error is thrown if i try to install the tarball that build.

Seeing this issue, i modified my pyproject.toml to use the poetry-core build-system and added the following lines. Now everything seems to be working again.

[tool.poetry.build]
generate-setup-file = false

Is this the intended behaviour? Is now the new method of building extensions?

[abn]

@ahobsonsayers you will still probably also need #2740 (comment).

[ahobsonsayers]

Sorry, yes, it seems I missed that line in my copy and paste of the lines. So is it intended for the script = "build.py" under [tool.poetry] to be deprecated and no longer work?