-
-
Notifications
You must be signed in to change notification settings - Fork 12
cutensor v2.1.0.9 #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 11 commits
ab441f0
612b2ea
af78451
f8827ce
df55175
8302649
3bcc3df
e932e16
6f8a131
fe95b3e
a85852b
398e193
86c88d4
94641d3
e67026a
aa8ae12
1ec96e7
09ed4af
b5b5ee0
aa21e4a
320d20b
2229b3f
9cbcf49
73002fb
940c41d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
This file was deleted.
This file was deleted.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
@echo on | ||
|
||
:: Hack to free up additional space on Azure | ||
:: Replace with conda-smithy solution when available | ||
:: xref: https://github.com/conda-forge/conda-smithy/pull/1966 | ||
if "%CI%" == "azure" ( | ||
set CLEANUP_DIRS=^ | ||
C:\hostedtoolcache\windows;^ | ||
; | ||
|
||
mkdir C:\empty | ||
for %%f in (%CLEANUP_DIRS:;= %) do ( | ||
if not [%%f] == [] ( | ||
echo Removing %%f | ||
dir %%f | ||
robocopy /mir /mt /zb /ns /nc /nfl /ndl /np /njh /njs C:\empty %%f > nul 2>&1 | ||
rmdir /q %%f | ||
) | ||
) | ||
rmdir /q C:\empty | ||
) | ||
|
||
if not exist %PREFIX% mkdir %PREFIX% | ||
if not exist %LIBRARY_PREFIX% mkdir %LIBRARY_PREFIX% | ||
if not exist %LIBRARY_BIN% mkdir %LIBRARY_BIN% | ||
if not exist %LIBRARY_LIB% mkdir %LIBRARY_LIB% | ||
if not exist %LIBRARY_INC% mkdir %LIBRARY_INC% | ||
|
||
copy include\cutensor.h %LIBRARY_INC%\ | ||
copy include\cutensorMg.h %LIBRARY_INC%\ | ||
mkdir %LIBRARY_INC%\cutensor | ||
copy include\cutensor\types.h %LIBRARY_INC%\cutensor\ | ||
del lib\{{ cuda_major }}\*static* | ||
copy lib\{{ cuda_major }}\*.dll %LIBRARY_BIN%\ | ||
copy lib\{{ cuda_major }}\*.lib %LIBRARY_LIB%\ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,11 @@ | ||
#!/usr/bin/env bash | ||
set -e | ||
|
||
check-glibc lib/$CUDA_MAJOR/*.so* | ||
|
||
export CUDA_MAJOR=${cuda_compiler_version%%.*} | ||
|
||
mkdir -p $PREFIX/include | ||
mv include/* $PREFIX/include/ | ||
mkdir -p $PREFIX/lib | ||
mv lib/$CUDA_MAJOR/*.so* $PREFIX/lib/ | ||
|
||
if [[ -f "${RECIPE_DIR}/common/detect-glibc" ]] ; then | ||
source "${RECIPE_DIR}/common/detect-glibc" | ||
fi | ||
|
||
SYSTEM_GLIBC_VERSION="$(glibc-detect system)" | ||
RECIPE_GLIBC_VERSION="${c_stdlib_version:=0.0.0}" | ||
|
||
for file in "${PREFIX}"/lib/libcutensor*.so.*; do | ||
if [[ -f "$file" && ! -L "$file" ]]; then # Ensure it's a file | ||
BINARY_GLIBC_VERSION="$(glibc-detect req $file)" | ||
echo "binary glibc ${BINARY_GLIBC_VERSION} <= recipe glibc ${RECIPE_GLIBC_VERSION} <= system glibc ${SYSTEM_GLIBC_VERSION} $file" | ||
BINARY_IS_COMPATIBLE="$(glibc-check compatible $BINARY_GLIBC_VERSION $RECIPE_GLIBC_VERSION)" | ||
if [[ $BINARY_IS_COMPATIBLE == "false" ]] ; then | ||
echo "The binary is not compatible with the recipe glibc pinning." | ||
exit 1 | ||
fi | ||
echo "The binary is compatible." | ||
fi | ||
done |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,2 @@ | ||
c_stdlib_version: # [linux] | ||
- "2.17" # [linux64] | ||
- "2.28" # [aarch64] | ||
- "2.28" # [ppc64le] | ||
- "2.28" # [linux] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have found
C:\hostedtoolcache\windows
takes up a fair bit of space on Azure. Removing it would free up that spaceHad put this solution together in this conda-smithy PR: conda-forge/conda-smithy#1966
Though have not found a good method for deleting this directory quickly on Windows. So the deletion takes a while. Maybe acceptable if there is no other option (like we see here)
That said, if we could find a faster way to do this deletion, it would be easier to integrate this approach in conda-smithy and just enable with a config parameter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which options have you tried?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did the bulk of the exploration in these PRs:
With matplotlib, it was picking up some VM Python during its build and getting hosed. We were not able to get the build to stop picking up the VM Python and couldn't figure out why it was showing up. So we went with the deletion option to get the build fixed up. Lots of commits in there trying things
Though running out of space is an often enough problem (like with this update), that I found the simplest out-of-the-way Windows only build and used that for testing, which is why MiKTeX was used. It was also simpler than going through a full matplotlib build each iteration
Tried different deletion methods. Found
robocopy
was better than the others. So then tweaked options to eek out more performanceIf you spot something I've missed, would be curious to learn about it