Skip to content

wheels: enforce tighter size limits in CI #219

@jameslamb

Description

@jameslamb

Description

RAPIDS 25.10 was the first release with CUDA 13 support (#208).

That support included changes to take advantage of improved fatbin compression options with nvcc 13:

As a result, for some libraries the CUDA 13 packages are now significantly smaller than the equivalent CUDA 12 libraries.

The size thresholds enforced in RAPIDS CI should be adjusted to account for this.

Benefits of this work

  • ensures we're able to catch unexpected binary size increases for wheels early, when it's easier to investigate where they came from and make informed decisions about whether the size increase is acceptable
  • helps keep RAPIDS wheels from growing too much, for all the reasons that's good (faster installs, lower storage footprint, reduced burden on package repositories, etc.)

Acceptance Criteria

  • for all RAPIDS projects, configured size thresholds in wheel-building CI are within some small amount (25 MiB?) of the current wheel sizes

Approach

Look in the output of wheel CI for lines similar to this:

----- package inspection summary -----
file size
  * compressed size: 0.5841G
  * uncompressed size: 0.9001G

or pip download nightlies and check the size yourself.

For projects where the wheel sizes don't differ by CUDA major version, continue to configure the sizes in pyproject.toml

[tool.pydistcheck]
select = [
    "distro-too-large-compressed",
]

# PyPI limit is 750 MiB, fail CI before we get too close to that
max_allowed_size_compressed = '675M'

(example from cudf)

For others, remove that max_allowed_size_compressed setting in pyproject.toml and instead pass different values via the command line. Something like this in ci/validate_wheel.sh:

RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"

if [[ "${package_dir}" == "python/libcuvs" ]]; then
    if [[ "${RAPIDS_CUDA_MAJOR}" == "12" ]]; then
        PYDISTCHECK_ARGS=(
            --max-allowed-size-compressed '500M'
        )
    else
        PYDISTCHECK_ARGS=(
            --max-allowed-size-compressed '300M'
        )
    fi
else
    PYDISTCHECK_ARGS=()
fi

cd "${package_dir}"

rapids-logger "validate packages with 'pydistcheck'"
pydistcheck \
    --inspect \
    "${PYDISTCHECK_ARGS[@]}" \
    "$(echo "${wheel_dir_relative_path}"/*.whl)"

(this is how these checks were configured when RAPIDS supported CUDA 11 and 12, see the diff of rapidsai/cuvs#960 for an example)

For all of them, choose a limit that is within some agreed amount (25 MiB?) of the current sizes.

Projects to check (GitHub search):

  • cucim
  • cudf
  • cugraph
  • cugraph-gnn
  • cuml
  • cuvs
  • cuxfilter
  • dask-cuda
  • jupyterlab-nvdashboard
  • kvikio
  • nx-cugraph
  • raft
  • rapidsmpf
  • rmm
  • ucxx

Notes

Size checks were originally introduced in #110

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions