Skip to content

Add more default python dist checks #481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Dec 22, 2022
Merged

Add more default python dist checks #481

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
dd2a5f0
add more default dist checks
blink1073 Dec 22, 2022
c0908da
update default
blink1073 Dec 22, 2022
e0a425e
fix default
blink1073 Dec 22, 2022
b6ade57
make script importable
blink1073 Dec 22, 2022
e5b7859
more cleanup
blink1073 Dec 22, 2022
71d2cfd
cleanup
blink1073 Dec 22, 2022
d041622
fixup
blink1073 Dec 22, 2022
0f60a9a
try again
blink1073 Dec 22, 2022
8ae372c
try again
blink1073 Dec 22, 2022
613a02b
spelling
blink1073 Dec 22, 2022
1a2658c
handle new asyncio behavior
blink1073 Dec 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ jobs:
run: |
set -eux
pip install -e .
python -m jupyter_releaser.actions.generate-changelog
python -m jupyter_releaser.actions.generate_changelog
cat CHANGELOG_ENTRY.md
# Check for version entry contents
cat CHANGELOG_ENTRY.md | grep -q "#234"
Expand All @@ -84,7 +84,7 @@ jobs:
export RH_CONVERT_TO_RST=true
sudo apt-get install pandoc
pip install pypandoc
python -m jupyter_releaser.actions.generate-changelog
python -m jupyter_releaser.actions.generate_changelog
cat CHANGELOG_ENTRY.md

test_minimum_versions:
Expand Down
0 ...er_releaser/actions/generate-changelog.py → ...er_releaser/actions/generate_changelog.py
View file
Open in desktop
File renamed without changes.
17 changes: 15 additions & 2 deletions jupyter_releaser/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,9 +289,19 @@ def main(force):
click.option(
"--pydist-check-cmd",
envvar="RH_PYDIST_CHECK_CMD",
default="pipx run twine check --strict",
default="pipx run twine check --strict {dist_file}",
help="The command to use to check a python distribution file",
),
click.option(
"--pydist-extra-check-cmds",
envvar="RH_EXTRA_PYDIST_CHECK_CMDS",
default=[
"pipx run 'validate-pyproject[all]' pyproject.toml",
Copy link
Member

@jtpio jtpio Dec 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this affect projects that use a root pyproject.toml for configuration and other metadata?

It looks like this might be a bit too strict in some cases. Is there a way to skip that check via the releaser config?

An example run where this affects the check release workflow: https://github.com/jupyterlite/jupyterlite/actions/runs/3764123821/jobs/6398242276

Copy link
Member

@jtpio jtpio Dec 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also seems to have an effect on JupyterLab, although probably good it catches potential issue there:

https://github.com/jupyterlab/jupyterlab/actions/runs/3764228169/jobs/6398454511

Copy link
Contributor Author

@blink1073 blink1073 Dec 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is targeting the pyproject.toml in the checkout dir. Here is an example config for ipykernel.

Copy link
Member

@jtpio jtpio Dec 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks for the example 👍

"pipx run check-wheel-contents --ignore W002 {dist_dir}",
],
multiple=True,
help="Extra checks to run against the pydist file",
),
click.option(
"--pydist-resource-paths",
envvar="RH_PYDIST_RESOURCE_PATHS",
Expand Down Expand Up @@ -446,7 +456,9 @@ def build_python(dist_dir, python_packages):
@add_options(check_imports_options)
@add_options(pydist_check_options)
@use_checkout_dir()
def check_python(dist_dir, check_imports, pydist_check_cmd, pydist_resource_paths):
def check_python(
dist_dir, check_imports, pydist_check_cmd, pydist_extra_check_cmds, pydist_resource_paths
):
"""Check Python dist files"""
for dist_file in glob(f"{dist_dir}/*"):
if Path(dist_file).suffix not in [".gz", ".whl"]:
Expand All @@ -457,6 +469,7 @@ def check_python(dist_dir, check_imports, pydist_check_cmd, pydist_resource_path
dist_file,
python_imports=check_imports,
check_cmd=pydist_check_cmd,
extra_check_cmds=pydist_extra_check_cmds,
resource_paths=pydist_resource_paths,
)

Expand Down
8 changes: 6 additions & 2 deletions jupyter_releaser/python.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,17 @@ def check_dist(
dist_file,
test_cmd="",
python_imports=None,
check_cmd="pipx twine check --strict",
check_cmd="pipx run twine check --strict {dist_file}",
extra_check_cmds=None,
resource_paths=None,
):
"""Check a Python package locally (not as a cli)"""
resource_paths = resource_paths or []
dist_file = util.normalize_path(dist_file)
util.run(f"{check_cmd} {dist_file}")
dist_dir = os.path.dirname(dist_file) # used for check cmds.

for cmd in [check_cmd] + list(extra_check_cmds or []):
util.run(cmd.format(**locals()))

test_commands = []

Expand Down
6 changes: 5 additions & 1 deletion jupyter_releaser/tee.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,11 @@ def run(args: Union[str, List[str]], **kwargs: Any) -> CompletedProcess:

check = kwargs.get("check", False)

loop = asyncio.get_event_loop_policy().get_event_loop()
try:
loop = asyncio.get_event_loop()
except Exception:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
result = loop.run_until_complete(_stream_subprocess(cmd, **kwargs))
atexit.register(loop.close)

Expand Down
1 change: 1 addition & 0 deletions jupyter_releaser/tests/test_cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ def test_list_envvars(runner):
post-version-message: RH_POST_VERSION_MESSAGE
post-version-spec: RH_POST_VERSION_SPEC
pydist-check-cmd: RH_PYDIST_CHECK_CMD
pydist-extra-check-cmds: RH_EXTRA_PYDIST_CHECK_CMDS
pydist-resource-paths: RH_PYDIST_RESOURCE_PATHS
python-packages: RH_PYTHON_PACKAGES
ref: RH_REF
Expand Down