Add `shellcheck` across `RAPIDS`

[gforsyth]

Description

shellcheck is a fast, static analysis tool for shell scripts. It's good at
flagging up unused variables, unintentional glob expansions, and other potential
execution and security headaches that arise from the wonders of bash (and other shlangs).

https://www.shellcheck.net/

Any shell script that is running in CI in a RAPIDS repo should be passing shellcheck.

Benefits of this work

• Fewer chances for shell-scripts to go sideways, or for shell scripts that
  complete to be conflated with shell scripts that have executed correctly.
• Reduction in manual review for the build team of new shell scripts or updated
  shell scripts in PRs, automating "linty" PR suggestions.

Notes

For the initial scope, I propose that we only run shellcheck on scripts in the
ci/ directory, or on any scripts that are called in GHA.

There are some repos with a lot of shell-scripts more related to local
build-process, and these are less critical to "get right" (although that can be
done incrementally in the future).

Further, making several small tweaks to existing local build scripts across
these packages may break things in subtle ways, so it's better to have buy-in
from project maintainers before delving too deeply beyond ci/

Acceptance Criteria

• shellcheck is run in CI against the ci/ directory in all RAPIDS repos that have shell scripts
• All shell scripts in-scope pass shellcheck or have explicit (individual)
  exceptions noted. (No blanket exemptions)

Approach

These repos can be updated in any order, since these updates are all self-contained.

The goal is to have each repo running shellcheck in GHA, similar to https://github.com/rapidsai/gha-tools/blob/main/.github/workflows/prs.yaml#L20-L24

• cucim (Add shellcheck to pre-commit and fix warnings cucim#814, refactor(shellcheck): fix all shellcheck warnings/errors cucim#890)
• cudf (Add shellcheck to pre-commit and fix warnings cudf#17778, refactor(shellcheck): fix all shellcheck warnings/errors cudf#19129)
• cugraph (Add shellcheck to pre-commit and fix warnings cugraph#4954)
• cugraph-docs (Add shellcheck to pre-commit and fix warnings cugraph-docs#80, refactor(shellcheck): fix all shellcheck warnings/errors cugraph-docs#131)
• cugraph-ops repo being archived
• cuml (Add shellcheck to pre-commit and fix warnings cuml#6246, refactor(shellcheck): fix all shellcheck warnings/errors cuml#6901)
• cuopt (rapidsai/cuopt#2197, refactor(shellcheck): fix all remaining shellcheck errors/warnings NVIDIA/cuopt#99)
• cuspatial (Add shellcheck to pre-commit and fix warnings cuspatial#1515)
• cuvs (Add shellcheck to pre-commit and fix warnings cuvs#662, refactor(shellcheck): fix all remaining warnings/errors cuvs#1019)
• cuxfilter (feat(shellcheck): add shellcheck and fix all warnings cuxfilter#698)
• dask-cuda (Add shellcheck to pre-commit and fix warnings dask-cuda#1427, refactor(shellcheck): enable for all files and fix remaining warnings dask-cuda#1512)
• dependency-file-generator (chore(ci): add shellcheck to pre-commit dependency-file-generator#139)
• integration (stop uploading to downloads.rapids.ai, add shellcheck integration#770)
• jupyterlab-nvdashboard (add shellcheck to pre-commit jupyterlab-nvdashboard#238)
• kvikio (Add shellcheck to pre-commit and fix warnings kvikio#621)
• legate-boost (isolate wheel-building in a shell script, run shellcheck legate-boost#127)
• legate-dataframe (add shellcheck to pre-commit legate-dataframe#54)
• legate-raft (add shellcheck to pre-commit legate-raft#20)
• nx-cugraph (Add shellcheck to pre-commit and fix warnings nx-cugraph#84)
• pre-commit-hooks (add shellcheck to pre-commit, simplify testing script pre-commit-hooks#69)
• pynvjitlink (Add shellcheck to pre-commit and fix warnings pynvjitlink#126)
• raft (Add shellcheck to pre-commit and fix warnings raft#2575, refactor(shellcheck): fix all remaining warnings/errors raft#2703)
• rapids-cli (added precommit and fixed linting errors rapids-cli#48)
• rapids-cmake (Add shellcheck to pre-commit and fix warnings rapids-cmake#772)
• rapids-dask-dependency (enforce pre-commit checks in CI, add verify-copyright hook rapids-dask-dependency#87)
• rapids-metadata (stop uploading to downloads.rapids.ai, add shellcheck rapids-metadata#54)
• rapidsmpf (feat(shellcheck): add pre-commit hook and fix all warnings and errors rapidsmpf#344)
• rmm (Add shellcheck to pre-commit and fix warnings rmm#1788) (refactor(shellcheck): enable for entire repo and fix all warnings rmm#1971)
• ucx-py (add shellcheck to pre-commit ucx-py#1145)
• ucx-wheels (Check CUDA symbols and build with CUDA 12.8.0 ucx-wheels#19)
• ucxx (Add shellcheck to pre-commit and fix warnings ucxx#381, fix remaining shellcheck warnings ucxx#441)
• wholegraph this is part of cugraph-gnn now

[jameslamb]

Hey thanks for putting this up! I totally support it.

But instead of invoking a separate GitHub action, I think we should do this via the shellcheck-py pre-commit hook.

repos:
  - repo: https://github.com/shellcheck-py/shellcheck-py
    rev: v0.10.0.1
    hooks:
      - id: shellcheck

I've been sprinkling that around some repos for the last couple months and have found it works great... lightweight to install, runs fast, exactly matches the behavior of other shellcheck packages like those from conda-forge or Homebrew.

For example:

https://github.com/rapidsai/deployment/blob/65e6acf22b7ade745c8822bda76bc535e4b53857/.pre-commit-config.yaml#L28

https://github.com/rapidsai/legate-boost/blob/b9462f8d77c2540ba44393bc04b9f75ad3103d9a/.pre-commit-config.yaml#L10-L13

Doing that via pre-commit means you can get local feedback instead of needing to wait for a CI run.

[gforsyth]

Sounds good to me -- do we also run pre-commit in CI to ensure files are compliant?

[jameslamb]

Yep!

By convention, every RAPIDS project uses a workflow called checks:

  checks:
    secrets: inherit
    needs: telemetry-setup
    uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@branch-25.02
    with:
      enable_check_generated_files: false
      ignored_pr_jobs: "telemetry-summarize"

(example from cudf)

Which looks for a script named ci/check_style.sh (code link in rapidsai/shared-workflows). (a bit of a misnomer because lots of static analysis catches more than just style, like correctness, efficiency, deprecations, etc., but anyway....)

Most repos then have content in that check_style.sh that runs pre-commit run --all-files (example from cudf).

So that all happens by convention but you can generally safely assume that every RAPIDS repo is running pre-commit run --all-files on every pull request.

[gforsyth]

I pushed up an example commit of what this might look like here for rmm: gforsyth/rmm@02f9758