ARROW-18101: [R] RecordBatchReaderHead from ExecPlan with UDF cannot be read #14518
Conversation
|
|
paleolimbot
force-pushed
the
r-exec-reader-head
branch
from
fcde0ad to
c9b04e2
Compare
|
Probably related existing failures related to |
paleolimbot
force-pushed
the
r-exec-reader-head
branch
from
c9b04e2 to
bd80201
Compare
|
This one is just waiting for #14582 before it can be revisited. |
paleolimbot
force-pushed
the
r-exec-reader-head
branch
from
3feeca0 to
8e09802
Compare
|
Benchmark runs are scheduled for baseline = 139a13e and contender = 63b91cc. 63b91cc is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
|
['Python', 'R'] benchmarks have high level of regressions. |
…be read (apache#14518) This PR enables the following to work: ``` r library(arrow, warn.conflicts = FALSE) #> Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information. register_scalar_function( "times_32", function(context, x) x * 32.0, int32(), float64(), auto_convert = TRUE ) record_batch(a = 1:1000) |> dplyr::mutate(b = times_32(a)) |> as_record_batch_reader() |> head(11) |> as_arrow_table() #> Table #> 11 rows x 2 columns #> $a <int32> #> $b <double> ``` (normally this isn't a problem because you should really call `head()` before `as_record_batch_reader()`, which communicates the top-k-ness of the query to the ExecPlan rather than relying on abandoning a RecordBatchReader; however, it still should work!) Authored-by: Dewey Dunnington <[email protected]> Signed-off-by: Dewey Dunnington <[email protected]>
3 participants
This PR enables the following to work:
(normally this isn't a problem because you should really call
head()beforeas_record_batch_reader(), which communicates the top-k-ness of the query to the ExecPlan rather than relying on abandoning a RecordBatchReader; however, it still should work!)