ARROW-15533: [C++] Check ARROW_WITH_OPENTELEMETRY in CI #12322

mbrobbel · 2022-02-02T17:13:26Z

Set ARROW_WITH_OPENTELEMETRY: ON in C++ GitHub Actions workflow
Add default value for ExecNode::ToStringExtra argument indent argument

github-actions · 2022-02-02T17:13:49Z

https://issues.apache.org/jira/browse/ARROW-15533

wjones127 · 2022-02-02T21:21:15Z

Is the option making it though to the build? I see the env variable but this line suggests it's not actually turned on in the CMake Configure:

https://github.com/apache/arrow/runs/5040342323?check_suite_focus=true#step:7:460

Also, maybe we should also add this to a Linux build, like the AMD64 Conda C++ one?

mbrobbel · 2022-02-03T08:30:37Z

Is the option making it though to the build? I see the env variable but this line suggests it's not actually turned on in the CMake Configure:

https://github.com/apache/arrow/runs/5040342323?check_suite_focus=true#step:7:460

Thanks for spotting that! I forgot to update the build script.

Also, maybe we should also add this to a Linux build, like the AMD64 Conda C++ one?

This is blocked on #11963. I'll add a reminder to that PR to enable OpenTelemetry in CI once that gets merged.

mbrobbel · 2022-02-03T09:23:47Z

Disabling OT in Windows builds in this PR. The fix from #11963 should also fix those builds: https://github.com/apache/arrow/blob/8573ba86990968ab95ac8e53d05071f917da93d4/cpp/cmake_modules/ThirdpartyToolchain.cmake#L4126-L4128

pitrou · 2022-02-03T11:27:38Z

@github-actions crossbow submit -g cpp

github-actions · 2022-02-03T11:28:54Z

Revision: 02edcb8

Submitted crossbow builds: ursacomputing/crossbow @ actions-1572

Task	Status
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-10-cpp-amd64
test-debian-10-cpp-i386
test-debian-11-cpp-amd64
test-debian-11-cpp-i386
test-fedora-33-cpp
test-ubuntu-18.04-cpp
test-ubuntu-18.04-cpp-release
test-ubuntu-18.04-cpp-static
test-ubuntu-20.04-cpp
test-ubuntu-20.04-cpp-14
test-ubuntu-20.04-cpp-17
test-ubuntu-20.04-cpp-bundled
test-ubuntu-20.04-cpp-thread-sanitizer
test-ubuntu-21.04-cpp

pitrou · 2022-02-03T12:40:29Z

@github-actions crossbow submit test-ubuntu-18.04-*

pitrou

+1, let's wait for CI

github-actions · 2022-02-03T12:41:38Z

Revision: 105b3c0

Submitted crossbow builds: ursacomputing/crossbow @ actions-1573

Task	Status
test-ubuntu-18.04-cpp
test-ubuntu-18.04-cpp-release
test-ubuntu-18.04-cpp-static
test-ubuntu-18.04-r-sanitizer

ursabot · 2022-02-03T13:31:17Z

Benchmark runs are scheduled for baseline = 56d060c and contender = 7d34a7c. 7d34a7c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.36% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.39% ⬆️0.78%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

jorisvandenbossche · 2022-02-04T11:38:48Z

The "python-sdist" nightly build started failing (see https://github.com/ursacomputing/crossbow/runs/5062761297?check_suite_focus=true), with a message that seems related to OpenTelemetry:

 -- stderr output is:
CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2421 (message):
  error: could not find git for clone of opentelemetry-proto
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
  cmake/opentelemetry-proto.cmake:9 (ExternalProject_Add)
  CMakeLists.txt:363 (include)



CMake Error at /build/cpp/opentelemetry_ep-prefix/src/opentelemetry_ep-stamp/opentelemetry_ep-configure-DEBUG.cmake:47 (message):
  Stopping after outputting logs.

So potentially related to this PR (note, I didn't actually look into it, just looked at commits in the last day that could be related to the new failure, and pinging here in case you could take a look).

mbrobbel · 2022-02-04T12:01:33Z

The "python-sdist" nightly build started failing (see https://github.com/ursacomputing/crossbow/runs/5062761297?check_suite_focus=true), with a message that seems related to OpenTelemetry:
 -- stderr output is:
CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2421 (message):
  error: could not find git for clone of opentelemetry-proto
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
  cmake/opentelemetry-proto.cmake:9 (ExternalProject_Add)
  CMakeLists.txt:363 (include)



CMake Error at /build/cpp/opentelemetry_ep-prefix/src/opentelemetry_ep-stamp/opentelemetry_ep-configure-DEBUG.cmake:47 (message):
  Stopping after outputting logs.
So potentially related to this PR (note, I didn't actually look into it, just looked at commits in the last day that could be related to the new failure, and pinging here in case you could take a look).

It looks like git is removed before the build starts: https://github.com/ursacomputing/crossbow/runs/5062761297?check_suite_focus=true#step:6:501.
Should we change that or disable OpenTelemetry for these builds?

jorisvandenbossche · 2022-02-04T12:11:00Z

This removal of git was added in #10342, cc @kszucs

pitrou · 2022-02-07T17:32:07Z

I opened #12356 for the python-sdist failure.

jorisvandenbossche · 2022-02-23T07:47:06Z

This removal of git was added in #10342, cc @kszucs

@kszucs do you remember the reason for it?
We now have a similar failure of the python-sdist build, this time the new substrait option requiring git:

 CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2421 (message):
  error: could not find git for clone of substrait_ep
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
  src/arrow/engine/CMakeLists.txt:50 (externalproject_add)

kszucs · 2022-02-23T07:55:21Z

This removal of git was added in #10342, cc @kszucs

@kszucs do you remember the reason for it?

Git was used to generate a version number even for a python sdist release with explicit version and already available source code.

We now have a similar failure of the python-sdist build, this time the new substrait option requiring git:

 CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2421 (message):
  error: could not find git for clone of substrait_ep
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
  src/arrow/engine/CMakeLists.txt:50 (externalproject_add)

I think we should avoid hard dependency on git and prefer downloading tarballs from github.

jorisvandenbossche · 2022-02-23T07:59:19Z

I think we should avoid hard dependency on git and prefer downloading tarballs from github.

Opened a JIRA for this: https://issues.apache.org/jira/browse/ARROW-15760

kszucs · 2022-02-23T08:01:57Z

We should pass

    URL "https://github.com/${ARROW_SUBSTRAIT_REPO}/archive/${ARROW_SUBSTRAIT_TAG}.tar.gz"

instead of

arrow/cpp/src/arrow/engine/CMakeLists.txt

Lines 51 to 52 in bee67c5

    
           GIT_REPOSITORY "${ARROW_SUBSTRAIT_REPO}" 
        
           GIT_TAG "${ARROW_SUBSTRAIT_TAG}"

mbrobbel added 2 commits February 2, 2022 18:10

Build Arrow with OpenTelemetry in C++ workflow

62e9bf0

Add default value for ExecNode::ToStringExtra indent argument

cf90cae

github-actions bot added the Component: C++ label Feb 2, 2022

mbrobbel changed the title ~~ARROW-15533: [C++] ARROW_WITH_OPENTELEMETRY is not checked in CI~~ ARROW-15533: [C++] Check ARROW_WITH_OPENTELEMETRY in CI Feb 2, 2022

mbrobbel added 2 commits February 2, 2022 22:52

Add ARROW_WITH_OPENTELEMETRY to C++ build script and Dockerfiles

6915ecd

Disable OpenTelemetry in conda build. Blocked on ARROW-15066.

ad470f6

Disable OpenTelemetry in Windows builds

0917b64

Enable OpenTelemetry in more builds

02edcb8

Disable on Ubuntu 18.04 (OpenTelemetry build failure)

105b3c0

pitrou approved these changes Feb 3, 2022

View reviewed changes

pitrou closed this in 7d34a7c Feb 3, 2022

mbrobbel deleted the arrow-15533 branch February 3, 2022 13:24

jorisvandenbossche mentioned this pull request Feb 23, 2022

MINOR: [CI][Python] Disable Arrow Engine (substrait) in the ubuntu-python-sdist-test build #12493

Closed

mbrobbel mentioned this pull request Feb 23, 2022

ARROW-15760: [C++] Avoid hard dependency on git in cmake (download tarballs from github instead) #12494

Closed

ARROW-15533: [C++] Check ARROW_WITH_OPENTELEMETRY in CI #12322

ARROW-15533: [C++] Check ARROW_WITH_OPENTELEMETRY in CI #12322

Uh oh!

Conversation

mbrobbel commented Feb 2, 2022

Uh oh!

github-actions bot commented Feb 2, 2022

Uh oh!

wjones127 commented Feb 2, 2022

Uh oh!

mbrobbel commented Feb 3, 2022

Uh oh!

mbrobbel commented Feb 3, 2022

Uh oh!

pitrou commented Feb 3, 2022

Uh oh!

github-actions bot commented Feb 3, 2022

Uh oh!

pitrou commented Feb 3, 2022

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 3, 2022

Uh oh!

ursabot commented Feb 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorisvandenbossche commented Feb 4, 2022

Uh oh!

mbrobbel commented Feb 4, 2022

Uh oh!

jorisvandenbossche commented Feb 4, 2022

Uh oh!

pitrou commented Feb 7, 2022

Uh oh!

jorisvandenbossche commented Feb 23, 2022

Uh oh!

kszucs commented Feb 23, 2022

Uh oh!

jorisvandenbossche commented Feb 23, 2022

Uh oh!

kszucs commented Feb 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ursabot commented Feb 3, 2022 •

edited

Loading