GH-43502: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43503

vibhatha · 2024-08-01T01:23:24Z

Rationale for this change

JNI tests had a typo in the script where we have included gandiva instead of dataset.

What changes are included in this PR?

This PR fixes that typo in the current version.

Are these changes tested?

Yes, by existing tests and CIs.

Are there any user-facing changes?

No

GitHub Issue: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43502
GitHub Issue: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43502

vibhatha · 2024-08-01T01:23:46Z

@github-actions crossbow submit -g java

github-actions · 2024-08-01T01:23:55Z

⚠️ GitHub issue #43502 has been automatically assigned in GitHub to PR creator.

github-actions · 2024-08-01T01:26:05Z

Revision: a39435082b2b676c76fae1e348192ea1965886a5

Submitted crossbow builds: ursacomputing/crossbow @ actions-a30ce59033

Task	Status
java-jars
test-conda-python-3.11-spark-master
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

jinchengchenghh · 2024-08-01T01:32:43Z

And it still has a problem , https://github.com/apache/arrow/blob/main/ci/docker/java-jni-manylinux-201x.dockerfile#L53, ARROW_JAVA_JNI set, so jni build exists, but jni test detect ARROW_DATASET

vibhatha · 2024-08-01T02:58:59Z

And it still has a problem , https://github.com/apache/arrow/blob/main/ci/docker/java-jni-manylinux-201x.dockerfile#L53, ARROW_JAVA_JNI set, so jni build exists, but jni test detect ARROW_DATASET

I didn't get it clearly, could you please elaborate a bit?

jinchengchenghh · 2024-08-01T03:19:35Z

It check the ARROW_DATASET, but it is OFF now, so as ARROW_GANDIVA https://github.com/apache/arrow/blob/main/ci/scripts/java_test.sh#L41

vibhatha · 2024-08-01T06:34:27Z

It check the ARROW_DATASET, but it is OFF now, so as ARROW_GANDIVA https://github.com/apache/arrow/blob/main/ci/scripts/java_test.sh#L41

Let me check, though if this is true, we have had the CIs running without any of these checks 🤔

vibhatha · 2024-08-01T06:35:33Z

@jinchengchenghh on top of that there is already a CI failure for dataset.

jinchengchenghh · 2024-08-01T06:43:24Z

It check the ARROW_DATASET, but it is OFF now, so as ARROW_GANDIVA https://github.com/apache/arrow/blob/main/ci/scripts/java_test.sh#L41

Let me check, though if this is true, we have had the CIs running without any of these checks 🤔

I think you are right, not sure where to set the environment, but the test really triggers.

I will try to fix the exception, my IDE sync cause some problems

jinchengchenghh · 2024-08-01T06:47:16Z

Can you mark it as ignore? I will draft a new PR to solve it.

vibhatha · 2024-08-01T06:47:17Z

It check the ARROW_DATASET, but it is OFF now, so as ARROW_GANDIVA https://github.com/apache/arrow/blob/main/ci/scripts/java_test.sh#L41

Let me check, though if this is true, we have had the CIs running without any of these checks 🤔

I think you are right, not sure where to set the environment, but the test really triggers.

I will take a look at that.

I will try to fix the exception, my IDE sync cause some problems

Thanks, that would be wonderful, but I am not sure if this is related to the recent PRs (didn't check yet).

vibhatha · 2024-08-01T06:48:06Z

Can you mark it as ignore? I will draft a new PR to solve it.

would you like to push the patch to this branch?

jinchengchenghh · 2024-08-01T06:48:26Z

#43506

vibhatha · 2024-08-01T06:48:45Z

Btw, does that failing test in the CI work locally?

jinchengchenghh · 2024-08-01T07:14:15Z

Do you know how to push to this branch? I don't think I have write access to vibhatha:gh-43502

jinchengchenghh · 2024-08-01T07:18:14Z

#41646 skip tests java-jars, maybe we still need environment ARROW_DATASET to trigger it in Java JNI / AMD64 manylinux2014 Java JNI test

vibhatha · 2024-08-01T07:41:07Z

Do you know how to push to this branch? I don't think I have write access to vibhatha:gh-43502

I will add the rights.

vibhatha · 2024-08-01T07:45:53Z

@jinchengchenghh I have added you and you should have commit rights, please check.

jinchengchenghh · 2024-08-01T07:46:06Z

ok thanks! I will draft a PR as soon as possible

jinchengchenghh · 2024-08-01T08:08:59Z

I investigate the question again, the exception should not occur, looks like the linked library is not right.

TestFragmentScanOptions.testCsvConvertOptions:80 expected: <[Id: Int(32, true), Name: Utf8, Language: Utf8]> but was: <[Id;Name;Language: Utf8]>

Can anyone help to check it？

vibhatha · 2024-08-01T08:10:18Z

I investigate the question again, the exception should not occur, looks like the linked library is not right.
TestFragmentScanOptions.testCsvConvertOptions:80 expected: <[Id: Int(32, true), Name: Utf8, Language: Utf8]> but was: <[Id;Name;Language: Utf8]>
Can anyone help to check it？

let me check this locally.

vibhatha · 2024-08-01T08:10:36Z

@github-actions crossbow submit java-jars

vibhatha · 2024-08-06T03:23:59Z

@github-actions crossbow submit -g java

github-actions · 2024-08-06T03:26:17Z

Revision: cb36191

Submitted crossbow builds: ursacomputing/crossbow @ actions-92c9a56dee

Task	Status
java-jars
test-conda-python-3.11-spark-master
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

vibhatha · 2024-08-06T08:00:59Z

@kou I updated as you suggested, also created an issue to track and update once the LLVM issue is resolved.

danepitkin · 2024-08-06T14:25:22Z

@github-actions crossbow submit java-jars

github-actions · 2024-08-06T14:27:33Z

Revision: cb36191

Submitted crossbow builds: ursacomputing/crossbow @ actions-67f1c3bfef

Task	Status
java-jars

vibhatha · 2024-08-07T01:56:49Z

@danepitkin shall we re-run the java-jars? I am not sure what that failure is.

kou · 2024-08-07T02:06:27Z

Hmm. self-hosted arm64 Linux runner may have a problem:

arrow/dev/tasks/java-jars/github.yml

Line 43 in 2e1b245

- runs_on: ["self-hosted", "Linux", "arm64"]

Disk size?

vibhatha · 2024-08-07T02:31:52Z

@kou honestly not sure...

danepitkin · 2024-08-07T17:34:22Z

I think the previous java-jars successful run proves its fine. However the Java JNI CI job is failing. Did we want to fix the TestCsvFragment options in this PR?

vibhatha · 2024-08-08T02:03:51Z

I think the previous java-jars successful run proves its fine. However the Java JNI CI job is failing. Did we want to fix the TestCsvFragment options in this PR?

Wasn't the objective in this PR though.

danepitkin · 2024-08-09T14:41:55Z

@github-actions crossbow submit java-jars

github-actions · 2024-08-09T14:44:12Z

Revision: cb36191

Submitted crossbow builds: ursacomputing/crossbow @ actions-f25c638877

Task	Status
java-jars

danepitkin

Gandiva builds, java-jars now failing on #43506

vibhatha · 2024-08-09T21:38:01Z

Thanks @danepitkin

conbench-apache-arrow · 2024-08-10T04:22:25Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit cdfdfb8.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

…st not test dataset module (apache#43503) ### Rationale for this change JNI tests had a typo in the script where we have included `gandiva` instead of `dataset`. ### What changes are included in this PR? This PR fixes that typo in the current version. ### Are these changes tested? Yes, by existing tests and CIs. ### Are there any user-facing changes? No * GitHub Issue: apache#43502 Authored-by: Vibhatha Abeykoon <[email protected]> Signed-off-by: Dane Pitkin <[email protected]>

vibhatha marked this pull request as ready for review August 1, 2024 01:23

vibhatha requested review from assignUser, jonkeane, kou and raulcd as code owners August 1, 2024 01:23

github-actions bot added the awaiting review Awaiting review label Aug 1, 2024

kou changed the title ~~GH-43502: [JAVA] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module~~ GH-43502: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module Aug 1, 2024

vibhatha mentioned this pull request Aug 1, 2024

GH-28866: [Java] Java Dataset API ScanOptions expansion #41646

Merged

vibhatha mentioned this pull request Aug 6, 2024

[Java] Gandiva Tests are failing due to linking issues apache/arrow-java#63

Closed

fix: disabling gandiva failing tests

cb36191

vibhatha force-pushed the gh-43502 branch from e8a8f94 to cb36191 Compare August 6, 2024 03:23

lidavidm approved these changes Aug 6, 2024

View reviewed changes

github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Aug 6, 2024

This was referenced Aug 6, 2024

MINOR: [Java] Bump org.apache.avro:avro from 1.11.3 to 1.12.0 in /java #43564

Merged

MINOR: [Java] Bump checker.framework.version from 3.45.0 to 3.46.0 in /java #43563

Merged

danepitkin approved these changes Aug 9, 2024

View reviewed changes

danepitkin merged commit cdfdfb8 into apache:main Aug 9, 2024

danepitkin removed the awaiting merge Awaiting merge label Aug 9, 2024

danepitkin mentioned this pull request Aug 9, 2024

[Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43502

Closed

jinchengchenghh mentioned this pull request Aug 12, 2024

[Java] Fix TestFragmentScanOptions test result does not match #43506

Closed

This was referenced Aug 12, 2024

GH-38255: [Java] Implement Flight SQL Bulk Ingestion #43551

Merged

GH-43506: [Java] Fix TestFragmentScanOptions result not match #43639

Merged

GH-43502: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43503

GH-43502: [Java] Fix Java JNI / AMD64 manylinux2014 Java JNI test not test dataset module #43503

Uh oh!

Conversation

vibhatha commented Aug 1, 2024 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

github-actions bot commented Aug 1, 2024

Uh oh!

github-actions bot commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

jinchengchenghh commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 1, 2024

Uh oh!

vibhatha commented Aug 6, 2024

Uh oh!

github-actions bot commented Aug 6, 2024

Uh oh!

vibhatha commented Aug 6, 2024

Uh oh!

danepitkin commented Aug 6, 2024

Uh oh!

github-actions bot commented Aug 6, 2024

Uh oh!

vibhatha commented Aug 7, 2024

Uh oh!

kou commented Aug 7, 2024

Uh oh!

vibhatha commented Aug 7, 2024

Uh oh!

danepitkin commented Aug 7, 2024

Uh oh!

vibhatha commented Aug 8, 2024

Uh oh!

danepitkin commented Aug 9, 2024

Uh oh!

github-actions bot commented Aug 9, 2024

Uh oh!

vibhatha commented Aug 1, 2024 •

edited by github-actions bot

Loading

jinchengchenghh commented Aug 1, 2024 •

edited

Loading