Skip to content

Conversation

@KyleFromNVIDIA
Copy link
Member

Use CUDA math wheels to reduce wheel size by not statically linking CUDA math libraries.

Contributes to rapidsai/build-planning#35

@KyleFromNVIDIA KyleFromNVIDIA requested review from a team as code owners August 20, 2024 13:51
@KyleFromNVIDIA KyleFromNVIDIA added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Aug 20, 2024
@KyleFromNVIDIA
Copy link
Member Author

Comparing 934645c to this branch with the cuvs wheel, Python 3.11, x86_64, CUDA 12...

Old

Build Time: 9m4s
Wheel Size: 1.13GB
Wheel Contents:

Archive:  cuvs_cu12-24.10.0a43-cp311-cp311-manylinux_2_28_x86_64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2024-08-20 17:31   cuvs_cu12.libs/
        0  2024-08-20 17:31   cuvs/
        0  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/
   253289  2024-08-20 17:31   cuvs_cu12.libs/libgomp-24e2ab19.so.1.0.0
        0  2024-08-20 17:31   cuvs/distance/
        0  2024-08-20 17:31   cuvs/common/
        0  2024-08-20 17:31   cuvs/test/
        0  2024-08-20 17:31   cuvs/neighbors/
      650  2024-08-20 17:31   cuvs/__init__.py
     1153  2024-08-20 17:31   cuvs/distance_type.pxd
1684918641  2024-08-20 17:31   cuvs/libcuvs.so
       11  2024-08-20 17:31   cuvs/VERSION
  1511281  2024-08-20 17:31   cuvs/libcuvs_c.so
       41  2024-08-20 17:31   cuvs/GIT_COMMIT
      986  2024-08-20 17:31   cuvs/_version.py
      693  2024-08-20 17:31   cuvs/distance/__init__.py
     1216  2024-08-20 17:31   cuvs/distance/distance.pxd
     4838  2024-08-20 17:31   cuvs/distance/distance.pyx
     1056  2024-08-20 17:31   cuvs/distance/CMakeLists.txt
        0  2024-08-20 17:31   cuvs/distance/__init__.pxd
    93377  2024-08-20 17:31   cuvs/distance/distance.cpython-311-x86_64-linux-gnu.so
      689  2024-08-20 17:31   cuvs/common/__init__.py
     1738  2024-08-20 17:31   cuvs/common/temp_raft.py
     3499  2024-08-20 17:31   cuvs/common/resources.pyx
     1083  2024-08-20 17:31   cuvs/common/CMakeLists.txt
        0  2024-08-20 17:31   cuvs/common/__init__.pxd
     3023  2024-08-20 17:31   cuvs/common/cydlpack.pyx
     1724  2024-08-20 17:31   cuvs/common/cydlpack.pxd
    41889  2024-08-20 17:31   cuvs/common/cydlpack.cpython-311-x86_64-linux-gnu.so
    58889  2024-08-20 17:31   cuvs/common/exceptions.cpython-311-x86_64-linux-gnu.so
     1164  2024-08-20 17:31   cuvs/common/exceptions.pyx
    84161  2024-08-20 17:31   cuvs/common/resources.cpython-311-x86_64-linux-gnu.so
      722  2024-08-20 17:31   cuvs/common/resources.pxd
     1128  2024-08-20 17:31   cuvs/common/c_api.pxd
     5983  2024-08-20 17:31   cuvs/test/test_brute_force.py
     5830  2024-08-20 17:31   cuvs/test/test_cagra.py
     4484  2024-08-20 17:31   cuvs/test/test_doctests.py
     2318  2024-08-20 17:31   cuvs/test/test_distance.py
     3976  2024-08-20 17:31   cuvs/test/test_ivf_flat.py
        0  2024-08-20 17:31   cuvs/test/__init__py
      345  2024-08-20 17:31   cuvs/test/test_version.py
     1167  2024-08-20 17:31   cuvs/test/ann_utils.py
     5700  2024-08-20 17:31   cuvs/test/test_ivf_pq.py
     7591  2024-08-20 17:31   cuvs/test/test_refine.py
     2206  2024-08-20 17:31   cuvs/test/test_serialization.py
        0  2024-08-20 17:31   cuvs/neighbors/ivf_flat/
        0  2024-08-20 17:31   cuvs/neighbors/filters/
        0  2024-08-20 17:31   cuvs/neighbors/ivf_pq/
        0  2024-08-20 17:31   cuvs/neighbors/brute_force/
        0  2024-08-20 17:31   cuvs/neighbors/cagra/
      794  2024-08-20 17:31   cuvs/neighbors/__init__.py
     1195  2024-08-20 17:31   cuvs/neighbors/CMakeLists.txt
        0  2024-08-20 17:31   cuvs/neighbors/__init__.pxd
     1222  2024-08-20 17:31   cuvs/neighbors/refine.pxd
     6241  2024-08-20 17:31   cuvs/neighbors/refine.pyx
   105337  2024-08-20 17:31   cuvs/neighbors/refine.cpython-311-x86_64-linux-gnu.so
      846  2024-08-20 17:31   cuvs/neighbors/ivf_flat/__init__.py
     1066  2024-08-20 17:31   cuvs/neighbors/ivf_flat/CMakeLists.txt
     3278  2024-08-20 17:31   cuvs/neighbors/ivf_flat/ivf_flat.pxd
    16407  2024-08-20 17:31   cuvs/neighbors/ivf_flat/ivf_flat.pyx
        0  2024-08-20 17:31   cuvs/neighbors/ivf_flat/__init__.pxd
   176513  2024-08-20 17:31   cuvs/neighbors/ivf_flat/ivf_flat.cpython-311-x86_64-linux-gnu.so
      695  2024-08-20 17:31   cuvs/neighbors/filters/__init__.py
     1085  2024-08-20 17:31   cuvs/neighbors/filters/CMakeLists.txt
        0  2024-08-20 17:31   cuvs/neighbors/filters/__init__.pxd
     2863  2024-08-20 17:31   cuvs/neighbors/filters/filters.pyx
      874  2024-08-20 17:31   cuvs/neighbors/filters/filters.pxd
    84289  2024-08-20 17:31   cuvs/neighbors/filters/filters.cpython-311-x86_64-linux-gnu.so
      844  2024-08-20 17:31   cuvs/neighbors/ivf_pq/__init__.py
     3754  2024-08-20 17:31   cuvs/neighbors/ivf_pq/ivf_pq.pxd
     1058  2024-08-20 17:31   cuvs/neighbors/ivf_pq/CMakeLists.txt
    20614  2024-08-20 17:31   cuvs/neighbors/ivf_pq/ivf_pq.pyx
        0  2024-08-20 17:31   cuvs/neighbors/ivf_pq/__init__.pxd
   184849  2024-08-20 17:31   cuvs/neighbors/ivf_pq/ivf_pq.cpython-311-x86_64-linux-gnu.so
      673  2024-08-20 17:31   cuvs/neighbors/brute_force/__init__.py
     1096  2024-08-20 17:31   cuvs/neighbors/brute_force/CMakeLists.txt
     1940  2024-08-20 17:31   cuvs/neighbors/brute_force/brute_force.pxd
     8783  2024-08-20 17:31   cuvs/neighbors/brute_force/brute_force.pyx
        0  2024-08-20 17:31   cuvs/neighbors/brute_force/__init__.pxd
   139081  2024-08-20 17:31   cuvs/neighbors/brute_force/brute_force.cpython-311-x86_64-linux-gnu.so
      865  2024-08-20 17:31   cuvs/neighbors/cagra/__init__.py
     3898  2024-08-20 17:31   cuvs/neighbors/cagra/cagra.pxd
     1065  2024-08-20 17:31   cuvs/neighbors/cagra/CMakeLists.txt
   184865  2024-08-20 17:31   cuvs/neighbors/cagra/cagra.cpython-311-x86_64-linux-gnu.so
        0  2024-08-20 17:31   cuvs/neighbors/cagra/__init__.pxd
    22045  2024-08-20 17:31   cuvs/neighbors/cagra/cagra.pyx
        0  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/licenses/
      118  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/WHEEL
     7011  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/RECORD
      850  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/METADATA
    11348  2024-08-20 17:31   cuvs_cu12-24.10.0a43.dist-info/licenses/LICENSE
---------                     -------
1688024003                     91 files

New

Build Time: 5m36s
Wheel Size: 607MB
Wheel Contents:

Archive:  cuvs_cu12-24.10.0a44-cp311-cp311-manylinux_2_28_x86_64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/
        0  2024-08-20 14:15   cuvs_cu12.libs/
        0  2024-08-20 14:15   cuvs/
        0  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/licenses/
      118  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/WHEEL
     7010  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/RECORD
      990  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/METADATA
    11348  2024-08-20 14:15   cuvs_cu12-24.10.0a44.dist-info/licenses/LICENSE
   253289  2024-08-20 14:15   cuvs_cu12.libs/libgomp-24e2ab19.so.1.0.0
        0  2024-08-20 14:15   cuvs/distance/
        0  2024-08-20 14:15   cuvs/common/
        0  2024-08-20 14:15   cuvs/test/
        0  2024-08-20 14:15   cuvs/neighbors/
      650  2024-08-20 14:15   cuvs/__init__.py
     1153  2024-08-20 14:15   cuvs/distance_type.pxd
891236721  2024-08-20 14:15   cuvs/libcuvs.so
       11  2024-08-20 14:15   cuvs/VERSION
  1511753  2024-08-20 14:15   cuvs/libcuvs_c.so
       41  2024-08-20 14:15   cuvs/GIT_COMMIT
      986  2024-08-20 14:15   cuvs/_version.py
      693  2024-08-20 14:15   cuvs/distance/__init__.py
     1216  2024-08-20 14:15   cuvs/distance/distance.pxd
     4838  2024-08-20 14:15   cuvs/distance/distance.pyx
     1056  2024-08-20 14:15   cuvs/distance/CMakeLists.txt
        0  2024-08-20 14:15   cuvs/distance/__init__.pxd
    93625  2024-08-20 14:15   cuvs/distance/distance.cpython-311-x86_64-linux-gnu.so
      689  2024-08-20 14:15   cuvs/common/__init__.py
     1738  2024-08-20 14:15   cuvs/common/temp_raft.py
     3499  2024-08-20 14:15   cuvs/common/resources.pyx
     1083  2024-08-20 14:15   cuvs/common/CMakeLists.txt
        0  2024-08-20 14:15   cuvs/common/__init__.pxd
     3023  2024-08-20 14:15   cuvs/common/cydlpack.pyx
     1724  2024-08-20 14:15   cuvs/common/cydlpack.pxd
    42137  2024-08-20 14:15   cuvs/common/cydlpack.cpython-311-x86_64-linux-gnu.so
    59137  2024-08-20 14:15   cuvs/common/exceptions.cpython-311-x86_64-linux-gnu.so
     1164  2024-08-20 14:15   cuvs/common/exceptions.pyx
    88505  2024-08-20 14:15   cuvs/common/resources.cpython-311-x86_64-linux-gnu.so
      722  2024-08-20 14:15   cuvs/common/resources.pxd
     1128  2024-08-20 14:15   cuvs/common/c_api.pxd
     5983  2024-08-20 14:15   cuvs/test/test_brute_force.py
     5830  2024-08-20 14:15   cuvs/test/test_cagra.py
     4484  2024-08-20 14:15   cuvs/test/test_doctests.py
     2318  2024-08-20 14:15   cuvs/test/test_distance.py
     3976  2024-08-20 14:15   cuvs/test/test_ivf_flat.py
        0  2024-08-20 14:15   cuvs/test/__init__py
      345  2024-08-20 14:15   cuvs/test/test_version.py
     1167  2024-08-20 14:15   cuvs/test/ann_utils.py
     5700  2024-08-20 14:15   cuvs/test/test_ivf_pq.py
     7591  2024-08-20 14:15   cuvs/test/test_refine.py
     2206  2024-08-20 14:15   cuvs/test/test_serialization.py
        0  2024-08-20 14:15   cuvs/neighbors/ivf_flat/
        0  2024-08-20 14:15   cuvs/neighbors/filters/
        0  2024-08-20 14:15   cuvs/neighbors/ivf_pq/
        0  2024-08-20 14:15   cuvs/neighbors/brute_force/
        0  2024-08-20 14:15   cuvs/neighbors/cagra/
      794  2024-08-20 14:15   cuvs/neighbors/__init__.py
     1195  2024-08-20 14:15   cuvs/neighbors/CMakeLists.txt
        0  2024-08-20 14:15   cuvs/neighbors/__init__.pxd
     1222  2024-08-20 14:15   cuvs/neighbors/refine.pxd
     6241  2024-08-20 14:15   cuvs/neighbors/refine.pyx
   105585  2024-08-20 14:15   cuvs/neighbors/refine.cpython-311-x86_64-linux-gnu.so
      846  2024-08-20 14:15   cuvs/neighbors/ivf_flat/__init__.py
     1066  2024-08-20 14:15   cuvs/neighbors/ivf_flat/CMakeLists.txt
     3278  2024-08-20 14:15   cuvs/neighbors/ivf_flat/ivf_flat.pxd
    16407  2024-08-20 14:15   cuvs/neighbors/ivf_flat/ivf_flat.pyx
        0  2024-08-20 14:15   cuvs/neighbors/ivf_flat/__init__.pxd
   176761  2024-08-20 14:15   cuvs/neighbors/ivf_flat/ivf_flat.cpython-311-x86_64-linux-gnu.so
      695  2024-08-20 14:15   cuvs/neighbors/filters/__init__.py
     1085  2024-08-20 14:15   cuvs/neighbors/filters/CMakeLists.txt
        0  2024-08-20 14:15   cuvs/neighbors/filters/__init__.pxd
     2863  2024-08-20 14:15   cuvs/neighbors/filters/filters.pyx
      874  2024-08-20 14:15   cuvs/neighbors/filters/filters.pxd
    88633  2024-08-20 14:15   cuvs/neighbors/filters/filters.cpython-311-x86_64-linux-gnu.so
      844  2024-08-20 14:15   cuvs/neighbors/ivf_pq/__init__.py
     3754  2024-08-20 14:15   cuvs/neighbors/ivf_pq/ivf_pq.pxd
     1058  2024-08-20 14:15   cuvs/neighbors/ivf_pq/CMakeLists.txt
    20614  2024-08-20 14:15   cuvs/neighbors/ivf_pq/ivf_pq.pyx
        0  2024-08-20 14:15   cuvs/neighbors/ivf_pq/__init__.pxd
   185097  2024-08-20 14:15   cuvs/neighbors/ivf_pq/ivf_pq.cpython-311-x86_64-linux-gnu.so
      673  2024-08-20 14:15   cuvs/neighbors/brute_force/__init__.py
     1096  2024-08-20 14:15   cuvs/neighbors/brute_force/CMakeLists.txt
     1940  2024-08-20 14:15   cuvs/neighbors/brute_force/brute_force.pxd
     8783  2024-08-20 14:15   cuvs/neighbors/brute_force/brute_force.pyx
        0  2024-08-20 14:15   cuvs/neighbors/brute_force/__init__.pxd
   139329  2024-08-20 14:15   cuvs/neighbors/brute_force/brute_force.cpython-311-x86_64-linux-gnu.so
      865  2024-08-20 14:15   cuvs/neighbors/cagra/__init__.py
     3898  2024-08-20 14:15   cuvs/neighbors/cagra/cagra.pxd
     1065  2024-08-20 14:15   cuvs/neighbors/cagra/CMakeLists.txt
   185113  2024-08-20 14:15   cuvs/neighbors/cagra/cagra.cpython-311-x86_64-linux-gnu.so
        0  2024-08-20 14:15   cuvs/neighbors/cagra/__init__.pxd
    22045  2024-08-20 14:15   cuvs/neighbors/cagra/cagra.pyx
---------                     -------
894353366                     91 files

Copy link
Member

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put up 2 small comments for your consideration. I'll return to this once our offline conversation on DLFW / devcontainers is resolved.

Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question but it's not blocking.

@KyleFromNVIDIA
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 1d6e8da into rapidsai:branch-24.10 Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci CMake improvement Improves an existing functionality non-breaking Introduces a non-breaking change Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants