Skip to content

Conversation

Krande
Copy link

@Krande Krande commented Apr 16, 2024

I thought I might give fortran support for windows a try here as well (seeing as we've had some success in adding fortran support on windows using flang in mumps, mgis, mfront).

However, it failed locally in my initial attempts on windows with Fortran compiler requires either intrinsic functions SIZEOF or STORAGE_SIZE

If that is not a quick fix, I can take the fortran support out and put it in a different PR.

Checklist

@Krande
Copy link
Author

Krande commented Apr 16, 2024

@conda-forge-admin, please rerender

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

conda-forge-webservices[bot] and others added 2 commits April 16, 2024 12:20
@Krande
Copy link
Author

Krande commented Apr 16, 2024

@conda-forge-admin, please rerender

@Krande
Copy link
Author

Krande commented Apr 17, 2024

Hey @h-vetinari, I tried flang-new here in hopes of compiling HDF5 with fortran enabled on windows.

Unfortunately it fails during the configuration :( Any ideas where I ought to start looking? (I have posted a question about this on the flang-compiler slack channel also).

-- The Fortran compiler identification is LLVMFlang 18.1.3
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: D:/bld/hdf5_1713270629406/_build_env/Library/bin/flang-new.exe - skipped
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Success
-- Performing Test H5_FORTRAN_HAVE_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_SIZEOF - Failed
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF - Failed
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE - Failed
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING - Failed
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE - Failed
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE - Failed
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE - Failed
-- Performing Test HAVE_ISO_FORTRAN_ENV
-- Performing Test HAVE_ISO_FORTRAN_ENV - Failed
CMake Error at config/cmake/HDF5UseFortran.cmake:128 (message):
  Fortran compiler requires either intrinsic functions SIZEOF or STORAGE_SIZE
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)

link to build logs

Based on

I had hopes that this could work given that it seems LLVM flang compiles HDF5 on linux.

@Krande
Copy link
Author

Krande commented Apr 18, 2024

@conda-forge-admin, please rerender

@Krande
Copy link
Author

Krande commented Apr 18, 2024

Okay, here's another update.

Regarding the flang windows error. It seems the previous reported error was a simple CMake config forcing invalid flang comands. I added a patch for that which seems to work.

The error we're seeing now is

-- The Fortran compiler identification is LLVMFlang 18.1.3
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: C:/Work/miniforge3/envs/msvc-flang/Library/bin/flang-new.exe - skipped
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Success
-- Performing Test H5_FORTRAN_HAVE_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_SIZEOF - Success
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF - Success
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE - Success
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING - Success
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE - Success
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE - Success
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE - Success
-- Performing Test HAVE_ISO_FORTRAN_ENV
-- Performing Test HAVE_ISO_FORTRAN_ENV - Success
-- Performing Test FORTRAN_CHAR_ALLOC
-- Performing Test FORTRAN_CHAR_ALLOC - Success
CMake Error at config/cmake/HDF5UseFortran.cmake:171 (list):
  list index: 2 out of range (-2, 1)
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)


CMake Error at config/cmake/HDF5UseFortran.cmake:178 (message):
  Failed to find available REAL KINDs for Fortran
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)


-- Configuring incomplete, errors occurred!

And by closer inspection of the CMakeConfigureLog.yaml

LINK: command "C:\\PROGRA~2\\MICROS~3\\2022\\BUILDT~1\\VC\\Tools\\MSVC\\1438~1.331\\bin\\Hostx64\\x64\\link.exe /nologo CMakeFiles\\cmTC_6dccc.dir\\testFortranCompiler1.f90.obj /out:cmTC_6dccc.exe /implib:cmTC_6dccc.lib /pdb:cmTC_6dccc.exe.dbg /version:0.0 /machine:x64 -stack:10000000 /INCREMENTAL:NO /subsystem:console ws2_32.lib wsock32.lib shlwapi.lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib/clang/18/lib/windows /MANIFEST:EMBED,ID=1" failed (exit code 1120) with the following output:
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECinteger_kinds referenced in function _QQmain
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECreal_kinds referenced in function _QQmain\x0d
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envEClogical_kinds referenced in function _QQmain\x0d
        cmTC_6dccc.exe : fatal error LNK1120: 3 unresolved externals\x0d
        ninja: build stopped: subcommand faile

it seems this might be related to llvm/llvm-project#77282.

@Krande
Copy link
Author

Krande commented Apr 26, 2024

Hey, here's a small update.

Once llvm/llvm-project#89403 is fixed it might be possible to compile HDF5 with fortran enabled on Windows using LLVM FLang. Related to HDFGroup/hdf5#4419

I'll just convert this PR to a draft for now (pending fix for LLVM flang).

@Krande Krande marked this pull request as draft April 26, 2024 06:59
@astrofrog astrofrog removed their request for review May 13, 2024 22:49
Krande added 2 commits July 31, 2024 09:35
…est-hdf5

# Conflicts:
#	.ci_support/osx_64_mpimpich.yaml
#	.ci_support/osx_64_mpinompi.yaml
#	.ci_support/osx_64_mpiopenmpi.yaml
#	recipe/bld.bat
#	recipe/conda_build_config.yaml
#	recipe/meta.yaml
@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Oct 12, 2024

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.
I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • Jinja2 variable references are suggested to take a {{<one space><variable name><one space>}} form. See lines [64].

@Krande
Copy link
Author

Krande commented Oct 12, 2024

@conda-forge-admin, please rerender

@Krande
Copy link
Author

Krande commented Oct 28, 2024

@conda-forge-admin, please rerender

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.

Here's what I've got...

For recipe/meta.yaml:

  • Recipe maintainer "hmaarrfk" does not exist
  • Recipe maintainer "jakirkham" does not exist
  • Recipe maintainer "gillins" does not exist
  • Recipe maintainer "groutr" does not exist
  • Recipe maintainer "ocefpaf" does not exist
  • Recipe maintainer "astrofrog" does not exist
  • Recipe maintainer "marcelotrevisani" does not exist
  • Recipe maintainer "scopatz" does not exist
  • Recipe maintainer "davidbrochart" does not exist
  • Recipe maintainer "SylvainCorlay" does not exist
  • Recipe maintainer "varlackc" does not exist
  • Recipe maintainer "zklaus" does not exist

For recipe/meta.yaml:

  • Jinja2 variable references are suggested to take a {{<one space><variable name><one space>}} form. See lines [64].

@Krande
Copy link
Author

Krande commented Oct 28, 2024

Hey @h-vetinari and @mjklemm. I thought I would revisit this.

Seeing as the HDF5 configuration still fails on the kinds check (see #217 (comment) for reference) on windows.

@mjklemm Did you manage to compile hdf5 on windows using a locally compiled version of flang? If so I might want to test compiling it for myself just to see what we're doing differently in our conda-forge build.

I thought I would simply skip this check by simply hardcoding the supported real and integers by flang. By doing something like this I managed to finish the configuration step, and start compiling!

if (CMAKE_Fortran_COMPILER MATCHES "flang")
    message(STATUS "PROG_OUTPUT=${PROG_OUTPUT}. Force override with \"1,4,8;4,8,16;33;3;3;4;1,4,8;\"")
    set(PROG_OUTPUT "1,4,8;4,8,16;33;3;3;4;1,4,8;")
else ()
    # Convert the string to a list of strings by replacing the carriage return with a semicolon
    string (REGEX REPLACE "[\r\n]+" ";" PROG_OUTPUT "${PROG_OUTPUT}")
endif ()

It runs for a bit until the compilation fails due to error: use of undeclared identifier 'real_C_LONG_DOUBLE_f'.
You guys got any ideas for how to get past this?

[401/3074] Building C object fortran\src\CMakeFiles\hdf5_f90cstub-shared.dir\H5_f.c.obj
FAILED: fortran/src/CMakeFiles/hdf5_f90cstub-shared.dir/H5_f.c.obj 
%BUILD_PREFIX%\Library\bin\clang-cl.exe  /nologo -DH5_BUILT_AS_DYNAMIC_LIB -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -Dhdf5_f90cstub_shared_EXPORTS -I%SRC_DIR%\src -I%SRC_DIR%\src\H5FDsubfiling -I%SRC_DIR%\build\src -I%SRC_DIR%\build\fortran -I%SRC_DIR%\build\fortran\shared -I%PREFIX%\Library\include -w  /DWIN32 /D_WINDOWS /O2 /Ob2 /DNDEBUG -MD -Wall -Warray-bounds -Wcast-qual -Wconversion -Wdouble-promotion -Wextra -Wformat=2 -Wframe-larger-than=16384 -Wimplicit-fallthrough -Wnull-dereference -Wunused-const-variable -Wwrite-strings -Wpedantic -Wvolatile-register-var -Wno-c++-compat /W3 /wd4100 /wd4706 /wd4127 /showIncludes /Fofortran\src\CMakeFiles\hdf5_f90cstub-shared.dir\H5_f.c.obj /Fdfortran\src\CMakeFiles\hdf5_f90cstub-shared.dir\ -c -- %SRC_DIR%\fortran\src\H5_f.c
%SRC_DIR%\fortran\src\H5_f.c(187,16): error: use of undeclared identifier 'real_C_LONG_DOUBLE_f'
  187 |     if (sizeof(real_C_LONG_DOUBLE_f) == sizeof(float)) {
      |                ^
%SRC_DIR%\fortran\src\H5_f.c(191,21): error: use of undeclared identifier 'real_C_LONG_DOUBLE_f'
  191 |     else if (sizeof(real_C_LONG_DOUBLE_f) == sizeof(double)) {
      |                     ^
%SRC_DIR%\fortran\src\H5_f.c(196,21): error: use of undeclared identifier 'real_C_LONG_DOUBLE_f'
  196 |     else if (sizeof(real_C_LONG_DOUBLE_f) == sizeof(long double)) {
      |                     ^
3 errors generated.
[402/3074] Building C object fortran\src\CMakeFiles\hdf5_f90cstub-shared.dir\H5Af.c.obj
[403/3074] Building C object fortran\src\CMakeFiles\hdf5_f90cstub-shared.dir\H5Df.c.obj
ninja: build stopped: subcommand failed.

from https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1065872&view=logs&j=0ad3a718-cdde-5a77-9020-8beac94f6974&t=eda7a7f8-f584-55f5-338d-6077df40dfcc&l=2005

Update:

Okay, I think I solved it. Or at least I managed to get past the error message by adding the following to H5f90.h

#ifndef real_C_LONG_DOUBLE_f
#define real_C_LONG_DOUBLE_f long double
#endif

The next error is related to BIND(C) statements. Hm, are BIND(C) not supported by flang on windows?

C:\bld\flang-split_1725521580766\work\flang\lib\Optimizer\CodeGen\Target.cpp:102: not yet implemented: passing VALUE BIND(C) derived type for this target
LLVM ERROR: aborting

from https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1065902&view=logs&j=0ad3a718-cdde-5a77-9020-8beac94f6974&t=eda7a7f8-f584-55f5-338d-6077df40dfcc&l=2058

Or maybe it's BIND(C) and the use of passing argument by value like so?

subroutine mysub(x) bind(c)
    integer, value :: x  ! x is passed by value, not by reference
end subroutine mysub

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Oct 28, 2024

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.
  • ℹ️ Jinja2 variable references are suggested to take a {{<one space><variable name><one space>}} form. See lines [65].

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/13397952320. Examine the logs at this URL for more detail.

@h-vetinari
Copy link
Member

The not yet implemented message is unfortunately a pretty clear indicator that said fortran language feature is not yet available in flang. You can retry with flang 20 in ~February/March.

@Krande
Copy link
Author

Krande commented Oct 29, 2024

@h-vetinari Yup, it looks like it. I can raise an issue over at the llvm-project to see if it is on their radar. In the meantime to progress development on windows fortran support for hdf5 I opened #237 where I try to use the latest m2w64 libs. It looks quite promising from the local tests I did.

@Krande
Copy link
Author

Krande commented Nov 1, 2024

@h-vetinari Seems like this is already fixed in the flang main branch (hopefully on track for v20). See llvm/llvm-project#114035 (comment). I'll try to test this asap. If it works, then we can consider issueing a pre-release of flang v20 on a non-main conda-forge label.

@h-vetinari
Copy link
Member

h-vetinari commented Nov 2, 2024

If it works, then we can consider issueing a pre-release of flang v20 on a non-main conda-forge label.

This is a nontrivial effort (not extreme either, but it needs a strong reason to do it). I probably won't get around to doing it, but if you want, you can do the following:

  • pick a commit on LLVM main that looks OK (highly subjective of course)
  • Take https://github.com/conda-forge/llvmdev-feedstock, merge main into dev, resolving all conflicts in favour of main (with the only exception of channel_targets: in CBC) then rerender
  • do something like conda-forge/llvmdev-feedstock@61a9a86 for the new commit you've chosen
  • open a PR that targets the dev branch and get it green (you might need to rebase the patches carried on the feedstock)
  • if it's green and looks OK, I'll merge
  • repeat the same exercise for all the components necessary until you get to flang-activation

You can expect this process to take at least a week (not work hours, but iterations to get things building and waiting for CI). I can help out if you're stuck somewhere, but that's about as much as I can promise for now.

@Krande
Copy link
Author

Krande commented Feb 18, 2025

Hey @h-vetinari,

I revisited the HDF5 fortran compilation on windows recently.

I compiled the latest LLVM Flang from the main branch to see what is fixed and what is not.
I thought I should try to compile flang the "normal" way (not split up into different conda packages) first to see what features exactly is missing from LLVM flang in order to compile HDF5,

As it turns out, LLVM Flang does not seem to be quite ready for the fortran compilation of HDF5 quite yet. See my latest comment -> llvm/llvm-project#114035 (comment).

On the plus side, the kinds check (identified in #217 (comment)) seems to be fixed.

So from what I can tell, Flang v20 at the moment should pass the cmake configuration step, but it will most likely fail compiling the full HDF5 code base.

…est-hdf5

# Conflicts:
#	.azure-pipelines/azure-pipelines-linux.yml
#	.azure-pipelines/azure-pipelines-osx.yml
#	.azure-pipelines/azure-pipelines-win.yml
#	.ci_support/linux_64_mpimpich.yaml
#	.ci_support/linux_64_mpinompi.yaml
#	.ci_support/linux_64_mpiopenmpi.yaml
#	.ci_support/linux_aarch64_mpimpich.yaml
#	.ci_support/linux_aarch64_mpinompi.yaml
#	.ci_support/linux_aarch64_mpiopenmpi.yaml
#	.ci_support/linux_ppc64le_mpimpich.yaml
#	.ci_support/linux_ppc64le_mpinompi.yaml
#	.ci_support/linux_ppc64le_mpiopenmpi.yaml
#	.ci_support/osx_64_mpimpich.yaml
#	.ci_support/osx_64_mpinompi.yaml
#	.ci_support/osx_64_mpiopenmpi.yaml
#	.ci_support/osx_arm64_mpimpich.yaml
#	.ci_support/osx_arm64_mpinompi.yaml
#	.ci_support/osx_arm64_mpiopenmpi.yaml
#	.scripts/build_steps.sh
#	.scripts/run_osx_build.sh
#	.scripts/run_win_build.bat
#	README.md
#	azure-pipelines.yml
#	recipe/conda_build_config.yaml
#	recipe/meta.yaml
@Krande
Copy link
Author

Krande commented Feb 18, 2025

@conda-forge-admin, please rerender

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants