Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ab441f0
updated v2.1.0.9
regro-cf-autotick-bot Jan 28, 2025
612b2ea
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
regro-cf-autotick-bot Jan 28, 2025
af78451
Update hashes
carterbox Jan 30, 2025
f8827ce
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
Jan 30, 2025
df55175
Use glibc 2.28 for all linux archs
carterbox Jan 30, 2025
8302649
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
Jan 30, 2025
3bcc3df
On Windows, switch to `bld.bat`
jakirkham Jan 31, 2025
e932e16
Add workaround to free up space on Azure image
jakirkham Jan 31, 2025
6f8a131
Constrain `patchelf` to `<0.18.0`
jakirkham Jan 31, 2025
fe95b3e
Move `patchelf` to `requirements/build`
jakirkham Jan 31, 2025
a85852b
Switch to `cf-nvidia-tools` for GLIBC check
jakirkham Jan 31, 2025
398e193
Set & use environment variable for `cuda_major`
jakirkham Jan 31, 2025
86c88d4
Sort `conda-forge.yml`'s keys
jakirkham Jan 31, 2025
94641d3
Simplify `skip`
jakirkham Jan 31, 2025
e67026a
Move Azure Windows cleanup into CI setup step
jakirkham Jan 31, 2025
aa8ae12
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
Jan 31, 2025
1ec96e7
Set `CLEANUP_DIRS` outside `if`
jakirkham Jan 31, 2025
09ed4af
Add `errorlevel` checks in `bld.bat`
jakirkham Jan 31, 2025
b5b5ee0
Fix indentation
jakirkham Jan 31, 2025
aa21e4a
Use `C:` on Azure Windows
jakirkham Jan 31, 2025
320d20b
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
Jan 31, 2025
2229b3f
Try removing Azure CI image custom cleanup step
jakirkham Jan 31, 2025
9cbcf49
MNT: Re-rendered with conda-build 25.1.1, conda-smithy 3.45.4, and co…
Jan 31, 2025
73002fb
Drop `patchelf` from `requirements/host`
jakirkham Jan 31, 2025
940c41d
Pin `cf-nvidia-tools` to `1`
jakirkham Jan 31, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .azure-pipelines/azure-pipelines-linux.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ c_compiler_version:
c_stdlib:
- sysroot
c_stdlib_version:
- '2.17'
- '2.28'
cdt_name:
- conda
channel_sources:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ c_compiler_version:
c_stdlib:
- sysroot
c_stdlib_version:
- '2.17'
- '2.28'
cdt_name:
- conda
channel_sources:
Expand Down

This file was deleted.

This file was deleted.

3 changes: 3 additions & 0 deletions .gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 0 additions & 14 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 35 additions & 0 deletions recipe/bld.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@echo on

:: Hack to free up additional space on Azure
:: Replace with conda-smithy solution when available
:: xref: https://github.com/conda-forge/conda-smithy/pull/1966
if "%CI%" == "azure" (
set CLEANUP_DIRS=^
C:\hostedtoolcache\windows;^
;

mkdir C:\empty
for %%f in (%CLEANUP_DIRS:;= %) do (
if not [%%f] == [] (
echo Removing %%f
dir %%f
robocopy /mir /mt /zb /ns /nc /nfl /ndl /np /njh /njs C:\empty %%f > nul 2>&1
rmdir /q %%f
)
)
rmdir /q C:\empty
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have found C:\hostedtoolcache\windows takes up a fair bit of space on Azure. Removing it would free up that space

Had put this solution together in this conda-smithy PR: conda-forge/conda-smithy#1966

Though have not found a good method for deleting this directory quickly on Windows. So the deletion takes a while. Maybe acceptable if there is no other option (like we see here)

That said, if we could find a faster way to do this deletion, it would be easier to integrate this approach in conda-smithy and just enable with a config parameter

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though have not found a good method for deleting this directory quickly on Windows.

Which options have you tried?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the bulk of the exploration in these PRs:

With matplotlib, it was picking up some VM Python during its build and getting hosed. We were not able to get the build to stop picking up the VM Python and couldn't figure out why it was showing up. So we went with the deletion option to get the build fixed up. Lots of commits in there trying things

Though running out of space is an often enough problem (like with this update), that I found the simplest out-of-the-way Windows only build and used that for testing, which is why MiKTeX was used. It was also simpler than going through a full matplotlib build each iteration

Tried different deletion methods. Found robocopy was better than the others. So then tweaked options to eek out more performance

If you spot something I've missed, would be curious to learn about it


if not exist %PREFIX% mkdir %PREFIX%
if not exist %LIBRARY_PREFIX% mkdir %LIBRARY_PREFIX%
if not exist %LIBRARY_BIN% mkdir %LIBRARY_BIN%
if not exist %LIBRARY_LIB% mkdir %LIBRARY_LIB%
if not exist %LIBRARY_INC% mkdir %LIBRARY_INC%

copy include\cutensor.h %LIBRARY_INC%\
copy include\cutensorMg.h %LIBRARY_INC%\
mkdir %LIBRARY_INC%\cutensor
copy include\cutensor\types.h %LIBRARY_INC%\cutensor\
del lib\{{ cuda_major }}\*static*
copy lib\{{ cuda_major }}\*.dll %LIBRARY_BIN%\
copy lib\{{ cuda_major }}\*.lib %LIBRARY_LIB%\
22 changes: 2 additions & 20 deletions recipe/build.sh
Original file line number Diff line number Diff line change
@@ -1,29 +1,11 @@
#!/usr/bin/env bash
set -e

check-glibc lib/$CUDA_MAJOR/*.so*

export CUDA_MAJOR=${cuda_compiler_version%%.*}

mkdir -p $PREFIX/include
mv include/* $PREFIX/include/
mkdir -p $PREFIX/lib
mv lib/$CUDA_MAJOR/*.so* $PREFIX/lib/

if [[ -f "${RECIPE_DIR}/common/detect-glibc" ]] ; then
source "${RECIPE_DIR}/common/detect-glibc"
fi

SYSTEM_GLIBC_VERSION="$(glibc-detect system)"
RECIPE_GLIBC_VERSION="${c_stdlib_version:=0.0.0}"

for file in "${PREFIX}"/lib/libcutensor*.so.*; do
if [[ -f "$file" && ! -L "$file" ]]; then # Ensure it's a file
BINARY_GLIBC_VERSION="$(glibc-detect req $file)"
echo "binary glibc ${BINARY_GLIBC_VERSION} <= recipe glibc ${RECIPE_GLIBC_VERSION} <= system glibc ${SYSTEM_GLIBC_VERSION} $file"
BINARY_IS_COMPATIBLE="$(glibc-check compatible $BINARY_GLIBC_VERSION $RECIPE_GLIBC_VERSION)"
if [[ $BINARY_IS_COMPATIBLE == "false" ]] ; then
echo "The binary is not compatible with the recipe glibc pinning."
exit 1
fi
echo "The binary is compatible."
fi
done
168 changes: 0 additions & 168 deletions recipe/common/detect-glibc

This file was deleted.

4 changes: 1 addition & 3 deletions recipe/conda_build_config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
c_stdlib_version: # [linux]
- "2.17" # [linux64]
- "2.28" # [aarch64]
- "2.28" # [ppc64le]
- "2.28" # [linux]
Loading
Loading