Add "devcontainers" dependency file key that excludes xgboost (#7006)

Unified conda devcontainers use the `all` file key to generate their dependency lists for each repository. Currently cuml includes `xgboost` in its `all` file key via the `test_python` dependency list. Unfortunately, this pulls in a conda dependency on `librmm` via `libxgboost`. This is problematic because we must build `librmm` from source in our unified devcontainers.

It seems like `xgboost` is a soft test dependency. In this PR, I add a new dependency file key called `devcontainers` that excludes `xgboost`. Developer conda environments, CI workflows, and pip devcontainers are unaffected by this change. Conda devcontainers will now require the user to manually install `xgboost` in order to run tests that need it, and this is noted in `BUILD.md`.

This is needed to solve the problems discussed in rapidsai/devcontainers#534 (comment).

Once this is merged, I will push a change to rapidsai/devcontainers#536 to use the new `devcontainers` file key.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Paul Taylor (https://github.com/trxcllnt)
  - James Lamb (https://github.com/jameslamb)
  - Simon Adorf (https://github.com/csadorf)

URL: #7006

Author: bdice

BUILD.md

@@ -94,6 +94,9 @@ If you want a list of the available Python tests:
 $ pytest cuML/tests --collect-only
 ```
 
+Note: To run tests requiring `xgboost` in conda devcontainers, users must install the `xgboost` conda package manually.
+See `dependencies.yaml` for more information.
+
 ### Manual Process
 
 Once dependencies are present, follow the steps below:

dependencies.yaml

@@ -28,6 +28,33 @@ files:
       - py_version
       - rapids_build_backend
       - test_python
+      - test_python_xgboost
+  devcontainers:
+    output: none
+    includes:
+      # "devcontainers" includes everything in "all", excluding test_python_xgboost
+      - common_build
+      - cuda
+      - cuda_version
+      - depends_on_cudf
+      - depends_on_cuda_python
+      - depends_on_cupy
+      - depends_on_cuvs
+      - depends_on_dask_cudf
+      - depends_on_libcumlprims
+      - depends_on_libcuvs
+      - depends_on_libraft
+      - depends_on_librmm
+      - depends_on_pylibraft
+      - depends_on_raft_dask
+      - depends_on_rapids_logger
+      - depends_on_rmm
+      - docs
+      - py_build_cuml
+      - py_run_cuml
+      - py_version
+      - rapids_build_backend
+      - test_python
   cpp_all:
     output: conda
     matrix:
@@ -83,6 +110,7 @@ files:
       - depends_on_libcuml
       - py_version
       - test_python
+      - test_python_xgboost
   test_notebooks:
     output: none
     includes:
@@ -146,6 +174,7 @@ files:
       key: test
     includes:
       - test_python
+      - test_python_xgboost
   py_build_libcuml:
     output: pyproject
     pyproject_dir: python/libcuml
@@ -415,11 +444,19 @@ dependencies:
           - pytest-xdist
           - seaborn
           - *scikit_learn
-          - &xgboost xgboost>=2.1.0
           - statsmodels
           - tenacity
           - umap-learn==0.5.7
           - pynndescent
+  test_python_xgboost:
+    common:
+      - output_types: [conda, requirements, pyproject]
+        packages:
+          # We must separate xgboost into its own list so that it is not
+          # included in the "devcontainers" key. The libxgboost package depends
+          # on librmm but we do not want to have a package depending on librmm
+          # in devcontainers since it should be built from source.
+          - &xgboost xgboost>=2.1.0
   test_notebooks:
     common:
       - output_types: [conda, requirements]