Arrow headers installed to /usr/local can break perspective's subdirectory Arrow build

[tomjakubowski]

Bug Report

Steps to Reproduce:

1. Build and install Apache Arrow to /usr/local
2. Try and build perspective, with its own included Arrow dependency

Expected Result:

Perspective builds Arrow with its own set of options (defined in FindInstallDependency.cmake)

Actual Result:

Flags from the global installation, defined in /usr/local/include/arrow/util/config.h, will interfere with perspective's Arrow build.

In my case, that global config file had #define ARROW_JEMALLOC, while perspective's build had (set ARROW_JEMALLOC OFF) in cmake, which are inconsistent flags and so Arrow was miscompiled with some missing symbols referencing their Jemalloc memory pool implementation.

nm -mg /Users/tom/perspective/perspective/rust/perspective-python/perspective/perspective.abi3.so | grep malloc

                 (undefined) external __ZN5arrow11memory_pool8internal17JemallocAllocator13ReleaseUnusedEv (dynamically looked up)
                 (undefined) external __ZN5arrow11memory_pool8internal17JemallocAllocator15AllocateAlignedExxPPh (dynamically looked up)
                 (undefined) external __ZN5arrow11memory_pool8internal17JemallocAllocator17DeallocateAlignedEPhxx (dynamically looked up)
                 (undefined) external __ZN5arrow11memory_pool8internal17JemallocAllocator17ReallocateAlignedExxxPPh (dynamically looked up)

Found this by running a make VERBOSE=1 from the arrow-build directory (at rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-build/). Logs below. Notice that the -I for /usr/local/include comes before the -I for arrow's own source file.

cd
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-build/src/arrow
&& /usr/bin/c++ -DARROW_WITH_TIMING_TESTS
-I/Users/tom/perspective/perspective/rust/perspective-server/cpp/perspective/src/include
-I/usr/local/include
-I/Users/tom/perspective/perspective/rust/perspective-server/cpp/perspective/../../python/perspective/perspective/perspective/include
-I/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-src/cpp/src
-I/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-src/cpp/src/generated
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/date-src/extras/date/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/date-src/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/date-src
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/hopscotch-src/extras/hopscotch/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/hopscotch-src/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/hopscotch-src
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/ordered-map-src/extras/ordered-map/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/ordered-map-src/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/ordered-map-src
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/rapidjson-src/extras/rapidjson/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/rapidjson-src/include
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/rapidjson-src
-isystem
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-build/src
-fno-aligned-new  -O3  -Qunused-arguments -fcolor-diagnostics  -Wall
-Wno-unknown-warning-option -Wno-pass-failed  -O3 -DNDEBUG -O2  -std=c++17
-arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk
-fPIC -MD -MT
arrow-build/src/arrow/CMakeFiles/arrow_memory_pool.dir/memory_pool.cc.o -MF
CMakeFiles/arrow_memory_pool.dir/memory_pool.cc.o.d -o
CMakeFiles/arrow_memory_pool.dir/memory_pool.cc.o -c
/Users/tom/perspective/perspective/rust/target/release/build/perspective-server-bb669fae7f7ee29c/out/build/arrow-src/cpp/src/arrow/memory_pool.cc

Environment:

macOS 13.3.1 perspective master, 13b26d7

Additional Context:

Here is the #include which was picking up from /usr/local instead of the build directory.

This seems like a potentially common hazard/footgun for people building from source, who may have an installed Arrow elsewhere on their system. I do not know if a libarrow installed to /usr (as might be done by a Linux distro package manager) causes the same problem but would be worth investigating.

[timkpaine]

this is a duplicate of many other similar issues:

• Can't install Perspective: Could not build wheels for perspective-python #1884
• Installation problem #1315

[texodus]

@timkpaine This is the tracking issue for a common, actual installation problem we intend to fix. For other Issues/Discussion that get opened about vague source installation problems that we think this is the fix for, let's close and reroute to this instead.

#1884 is a Discussion not an Issue - and the mention related to this issue is an unrelated necropost. Let's close and re-route these types to the Issues we are going to resolve to address them if we can.
#1315 does not look relevant to this path error - however, the point of opening up this (and future) Issues is to quantize where our source installation process can be discretely improved.

[timkpaine]

issue for system-installed deps conflicting with the build, you converted it to a discussion from this issue: #1839

[timkpaine]

I think the problem is (unsurprisingly) CMake uncleanliness again:

perspective/cpp/perspective/CMakeLists.txt

Line 317 in 13b26d7

include_directories("/usr/local/include")

[texodus]

That's not an actionable Issue either (bug template), but if the discrete resolution to #1839 is a fix for this, let's reroute to this Issue in the comments also please.