[R] Package v21.0.0 not building on Debian 12 with CMake installed via APT

[benz0li]

Describe the bug, including details regarding any error messages, version, and platform.

My (multi-arch x86_64, AArch64) pipelines fail with v21.0.0: https://gitlab.b-data.ch/jupyterlab/r/tidyverse/-/jobs/189681.
ℹ️ This pipeline builds container images based on debian:12.

I had to roll back to v20.0.0.2.

Raw logs: https://gitlab.b-data.ch/jupyterlab/r/tidyverse/-/jobs/189681/raw

---

r/NEWS.md states

• Require CMake 3.25 or greater in bundled build script for full-source builds (GH-46831: [C++][R] Remove some pending references to CMake < 3.25 (docs + minor CMake references) #46834). This shouldn't affect most users.

My pipelines' build log states

CMake Error at cmake_modules/ThirdpartyToolchain.cmake:1739 (message):
Require CMake 3.26 or later for building bundled Apache Thrift

ℹ️ The Debian 12 [package] repository provides CMake 3.25.1.

---

Cross reference:

• GH-47120: [R] Update NEWS for 21.0.0 #47121

Ping @amoeba

Component(s)

R

[benz0li]

ℹ️ It builds fine on Ubuntu 22.04 (Base image: nvidia/cuda:12.9.1-devel-ubuntu22.04; x86_64; CMake 3.22.1): https://gitlab.b-data.ch/jupyterlab/cuda/r/tidyverse/-/jobs/189602

Raw logs: https://gitlab.b-data.ch/jupyterlab/cuda/r/tidyverse/-/jobs/189602/raw

[thisisnic]

Look at your build logs, on Debian, the Arrow C++ library (libarrow) is being built from source, whereas on Ubuntu it's using a prebuilt binary. The compilation of the binary is where the CMake version error is coming from.

I think what's happened is that the instruction to update CMake to 3.25 was correct when we changed that (#46831) until we upgraded our bundled version of Thrift recently (#46740) at which point we needed to bump CMake to 3.26.

So we need to update references to CMake 3.25 to instead say CMake 3.26 across the project (including but not limited to the R package). For R, we have a few changes we need to make anyway relating to the recent release, so we can include this here too.

Maybe we want some script (as part of the release process?) which checks for the CMake version and flags it up if we need to bump this in our docs.

I'm a bit confused as it looks like packages are being installed from Posit Public Package manager (p3m) which AFAIK ships the full binaries including the C++ binary for Linux and does have them for Debian.

@benz0li - if you're happy to wait for the 21.0.0.1 release, this should be fixed in there. I'm still unsure why it failed as I'd expect the Debian installation to have used the C++ binary, but I'm wondering if libcurl or openssl is missing? If you wanted 21.0.0 sooner, there's more info in this bit of the docs on how to control source/binary installation of libarrow on Linux: https://arrow.apache.org/docs/r/articles/install.html

[kou]

To use the Arrow C++ binary on Debian: We may need export LIBARROW_BINARY=true by users or need to add debian to https://github.com/apache/arrow/blob/main/r/tools/nixlibs-allowlist.txt .

To build the Arrow C++ source on Debian 12: Could you install libthrift-dev in https://gitlab.b-data.ch/jupyterlab/r/tidyverse/-/blob/main/tidyverse/latest.Dockerfile?ref_type=heads to avoid building bundled Apache Thrift that requires CMake 3.26 or later?

[benz0li]

I'm a bit confused as it looks like packages are being installed from Posit Public Package manager (p3m) which AFAIK ships the full binaries including the C++ binary for Linux and does have them for Debian.

p3m offers source packages, too. That is what is being used.

I'm still unsure why it failed as I'd expect the Debian installation to have used the C++ binary, but I'm wondering if libcurl or openssl is missing?

Both libcurl4-openssl-dev and libssl-dev are installed.

[benz0li]

To reproduce (con x86_64): docker run --rm -ti glcr.b-data.ch/r/base bash -c 'ARROW_R_DEV=true && apt-get update && apt-get -y install --no-install-recommends cmake && install2.r --error --skipinstalled arrow'

debian12_arrow_build-logs.txt

[benz0li]

What surprises me:

On Debian 12 (x86_64)¹:

...
* installing *source* package ‘arrow’ ...
** this is package ‘arrow’ version ‘21.0.0’
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** pkg-config found.
trying URL 'https://apache.jfrog.io/artifactory/arrow/r/'
downloaded 1500 bytes

*** Found local C++ source: 'tools/cpp'
*** Building libarrow from source
    For build options and troubleshooting, see the install guide:
    https://arrow.apache.org/docs/r/articles/install.html
*** Building with MAKEFLAGS=-j2
**** cmake 3.25.1: /usr/bin/cmake
**** arrow with SOURCE_DIR='tools/cpp' BUILD_DIR='/tmp/RtmpBQakSl/filec8221997587' DEST_DIR='libarrow/arrow-21.0.0' CMAKE='/usr/bin/cmake' EXTRA_CMAKE_FLAGS='' CC='gcc' CXX='g++ -std=gnu++17' LDFLAGS='-L/usr/local/lib' N_JOBS='2' ARROW_S3='OFF' ARROW_GCS='OFF'
...

On Ubuntu 22.04 (x86_64)²:

...
* installing *source* package ‘arrow’ ...
** this is package ‘arrow’ version ‘21.0.0’
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** pkg-config found.
trying URL 'https://apache.jfrog.io/artifactory/arrow/r/'
downloaded 1500 bytes

*** Found libcurl and OpenSSL >= 3.0.0
trying URL 'https://apache.jfrog.io/artifactory/arrow/r/21.0.0/libarrow/bin/linux-openssl-3.0/arrow-21.0.0.zip'
Content type 'application/zip' length 42677997 bytes (40.7 MB)
==================================================
downloaded 40.7 MB

*** Checksum validated successfully for libarrow
*** Successfully retrieved libarrow (linux-openssl-3.0)
PKG_CFLAGS=-DARROW_DS_STATIC -DPARQUET_STATIC -DARROW_ACERO_STATIC -DARROW_COMPUTE_STATIC -DARROW_STATIC -I/tmp/RtmpSmtL4C/R.INSTALLc431f7ddef2/arrow/libarrow/arrow-21.0.0/include -I/usr/lib/include/x86_64-linux-gnu -I/usr/lib/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_COMPUTE -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS
PKG_LIBS=-L/tmp/RtmpSmtL4C/R.INSTALLc431f7ddef2/arrow/libarrow/arrow-21.0.0/lib -L/usr/lib/lib/x86_64-linux-gnu -larrow_dataset -lparquet -larrow_acero -larrow_compute -larrow -larrow_bundled_dependencies -ldl -lcurl -lssl -lcrypto
...

Footnotes

1. docker run --rm -ti glcr.b-data.ch/r/base bash -c 'ARROW_R_DEV=true && apt-get update && apt-get -y install --no-install-recommends cmake && install2.r --error --skipinstalled arrow' ↩

2. docker run --rm -ti glcr.b-data.ch/cuda/r/base bash -c 'ARROW_R_DEV=true && apt-get update && apt-get -y install --no-install-recommends cmake && install2.r --error --skipinstalled arrow' ↩

[benz0li]

On Debian 12 (x86_64)¹:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libcurl4-openssl-dev/now 7.88.1-10+deb12u12 amd64 [installed,local]
libssl-dev/now 3.0.16-1~deb12u1 amd64 [installed,local]
libssl3/now 3.0.16-1~deb12u1 amd64 [installed,local]
openssl/now 3.0.16-1~deb12u1 amd64 [installed,local]

On Ubuntu 22.04 (x86_64)²:

==========
== CUDA ==
==========

CUDA Version 12.9.1

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.


WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libcurl4-openssl-dev/now 7.81.0-1ubuntu1.20 amd64 [installed,local]
libssl-dev/now 3.0.2-0ubuntu1.19 amd64 [installed,local]
libssl3/now 3.0.2-0ubuntu1.19 amd64 [installed,local]
openssl/now 3.0.2-0ubuntu1.19 amd64 [installed,local]

Footnotes

1. docker run --rm -ti glcr.b-data.ch/r/base bash -c 'apt list --installed | grep ssl' ↩

2. docker run --rm -ti glcr.b-data.ch/cuda/r/base bash -c 'apt list --installed | grep ssl' ↩

[benz0li]

And yes, docker run --rm -ti glcr.b-data.ch/r/base bash -c 'install2.r --error --skipinstalled arrow', i.e. not installing CMake via APT, works.

For both x86_64 and AArch64. But it takes a very long time.