[R][CI] Segfault during Python flight tests

[jonkeane]

Describe the bug, including details regarding any error messages, version, and platform.

This is not consistent (re-runs will sometimes resolve it), but we seem to be getting a segfault when we are running our python-flight tests in R:

Start test: flight_put with overwrite = FALSE
  'test-python-flight.R:66' [success]
/arrow/cpp/src/arrow/status.cc:134: Invalid: Signal stop source already set up

 *** caught segfault ***
address 0xb8, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)

1 error ✖ | 0 warnings ✔ | 2 notes ✖
1
Error: `docker-compose --file /home/runner/work/arrow/arrow/docker-compose.yml run --rm -e TZ=MART -e ARROW_R_FORCE_TESTS=true ubuntu-r` exited with a non-zero exit code 1, see the process log above.

from: https://github.com/apache/arrow/actions/runs/3851391995/jobs/6562508211#step:7:24718

I suspect that this is the code that is segfaulting (based on the test that immediately preceded it):

arrow/r/tests/testthat/test-python-flight.R

Lines 70 to 72 in 6bd847b

	# Default is TRUE so this will overwrite
	flight_put(client, example_with_times, path = flight_obj)
	expect_identical(as.data.frame(flight_get(client, flight_obj)), example_with_times)

Some hypotheses from @paleolimbot :

It is almost certainly the cancellation stuff that was updated in R and C++ between the last release and now. Is there any chance that the Python arrow that's being used is outdated?
Dewey Dunnington: That seems new...I'm getting it in CI now and I didn't get before (and the PR that I'm thinking of has been merged for a while)
Dewey Dunnington: The SignalStopSource already set up message is because R's cancellation trumps Python's and so we get a warning. The segfault, however, is definitely new.

Component(s)

Continuous Integration, R

[paleolimbot]

Some cancellation behaviour was changed on the C++ side in #14250, although I don't know that it's necessarily related to a segfault in using flight. That could in theory be related to some symbol collision from loading common libraries, too.

[thisisnic]

I believe this has been fixed by #35812 but feel free to reopen if not