Skip to content

ARROW-6871: [Java] Enhance TransferPair related parameters check and tests #5645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tianchen92 wants to merge 4 commits into from
Closed

ARROW-6871: [Java] Enhance TransferPair related parameters check and tests #5645

tianchen92 wants to merge 4 commits into from

Conversation

@tianchen92
Copy link
Contributor

@tianchen92 tianchen92 commented Oct 14, 2019 •
edited
Loading

Related to ARROW-6871.

TransferPair related param checks in different classes have potential problems:

i. splitAndTansfer has no indices check in classes like VarcharVector
ii. splitAndTranser indices check in classes like UnionVector is not correct (Preconditions.checkArgument(startIndex + length <= valueCount)), should check params separately.
iii. should add more UT to cover corner cases.

@tianchen92
Copy link
Contributor Author

tianchen92 commented Oct 14, 2019

cc @siddharthteotia

@github-actions
Copy link

github-actions bot commented Oct 14, 2019

https://issues.apache.org/jira/browse/ARROW-6871

@tianchen92 tianchen92 closed this Oct 14, 2019
@tianchen92 tianchen92 reopened this Oct 14, 2019
@tianchen92 tianchen92 closed this Oct 14, 2019
@tianchen92 tianchen92 reopened this Oct 14, 2019
@tianchen92
Copy link
Contributor Author

tianchen92 commented Oct 23, 2019

also cc @praveenbingo appreciate if you have time to take a look.

liyafan82
liyafan82 reviewed Nov 19, 2019
View reviewed changes
@emkornfield
Copy link
Contributor

emkornfield commented Jan 11, 2020

@tianchen92 did you add the benchmark you mentioned in the JIRA?

@tianchen92
Copy link
Contributor Author

tianchen92 commented Jan 11, 2020 •
edited
Loading

@tianchen92 did you add the benchmark you mentioned in the JIRA?

I added a benchmark for copyValueSafe and shows no regression, however, due to #5645 (comment), I finally revert the check in copyValueSafe API.
Need to add benchmark for other API related to this PR?

@tianchen92
Copy link
Contributor Author

tianchen92 commented Jan 17, 2020

I add a benchmark and shows no regression.

Before:
Benchmark Mode Cnt Score Error Units
TransferPairBenchmarks.splitAndTransferIntVector avgt 5 1.801 ± 0.119 us/op
TransferPairBenchmarks.splitAndTransferVarcharVector avgt 5 12.794 ± 0.812 us/op

After:
Benchmark Mode Cnt Score Error Units
TransferPairBenchmarks.splitAndTransferIntVector avgt 5 1.800 ± 0.102 us/op
TransferPairBenchmarks.splitAndTransferVarcharVector avgt 5 12.570 ± 1.339 us/op

@emkornfield
Copy link
Contributor

emkornfield commented Feb 3, 2020

+1 thanks.

kszucs pushed a commit that referenced this pull request Feb 7, 2020
@tianchen92 @kszucs
ARROW-6871: [Java] Enhance TransferPair related parameters check and …
819b18d 
…tests

Related to [ARROW-6871](https://issues.apache.org/jira/browse/ARROW-6871).

TransferPair related param checks in different classes have potential problems:

i. splitAndTansfer has no indices check in classes like VarcharVector
ii. splitAndTranser indices check in classes like UnionVector is not correct (Preconditions.checkArgument(startIndex + length <= valueCount)), should check params separately.
iii. should add more UT to cover corner cases.

Closes #5645 from tianchen92/ARROW-6871 and squashes the following commits:

f3b897d <tianchen> fix style
0d3c7ea <tianchen> add benchmark
01f9a48 <tianchen> revert changes in copyFrom
a22d58a <tianchen> ARROW-6871:  Enhance TransferPair related parameters check and tests

Authored-by: tianchen <[email protected]>
Signed-off-by: Micah Kornfield <[email protected]>
@asfimport asfimport mentioned this pull request Feb 3, 2020
Closed
pribor pushed a commit to GlobalWebIndex/arrow that referenced this pull request Oct 24, 2025
@tianchen92 @kou
ARROW-6871: [Java] Enhance TransferPair related parameters check and …
603f5da 
…tests

Related to [ARROW-6871](https://issues.apache.org/jira/browse/ARROW-6871).

TransferPair related param checks in different classes have potential problems:

i. splitAndTansfer has no indices check in classes like VarcharVector
ii. splitAndTranser indices check in classes like UnionVector is not correct (Preconditions.checkArgument(startIndex + length <= valueCount)), should check params separately.
iii. should add more UT to cover corner cases.

Closes apache#5645 from tianchen92/ARROW-6871 and squashes the following commits:

f3b897d <tianchen> fix style
0d3c7ea <tianchen> add benchmark
01f9a48 <tianchen> revert changes in copyFrom
a22d58a <tianchen> ARROW-6871:  Enhance TransferPair related parameters check and tests

Authored-by: tianchen <[email protected]>
Signed-off-by: Micah Kornfield <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liyafan82 liyafan82 liyafan82 left review comments

@emkornfield emkornfield Awaiting requested review from emkornfield

Assignees

No one assigned

Labels

Component: Java

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tianchen92 @emkornfield @liyafan82