Skip to content

Conversation

@paleolimbot
Copy link
Member

@paleolimbot paleolimbot commented Nov 9, 2022

This fixes test failures resulting from an updated lubridate package, whose update removed some functionality that we supported (apparently lubridate::yq(2021.1) was a thing). This works in our source because we cast to string() before doing any further processing.

@github-actions
Copy link

github-actions bot commented Nov 9, 2022

@github-actions
Copy link

github-actions bot commented Nov 9, 2022

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@nealrichardson
Copy link
Member

On the current test error, it looks like some of the floats do get handled by lubridate, only 2 of the values don't parse--so is removal the right fix?

@paleolimbot
Copy link
Member Author

Ah good point, it's the years with the trailing zero:

lubridate::parse_date_time(c(3.2007, 2.1970, 1.2020, 4.2009, 1.1975, NA), "qY")
#> Warning: 2 failed to parse.
#> [1] "2007-07-01 UTC" NA               NA               "2009-10-01 UTC"
#> [5] "1975-01-01 UTC" NA

Is it worth the maintenance effort to support numeric inputs here?

Copy link
Member

@rok rok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yq/Yq on string still seems to be a thing? https://github.com/tidyverse/lubridate/blob/main/tests/testthat/test-parsers.R#L746-L753
I think the float option is odd but don't have strong feelings on keeping/cutting it.

@rok
Copy link
Member

rok commented Nov 9, 2022

Sorry I misread the float/string part.

@nealrichardson
Copy link
Member

I agree it's odd but it does seem like lubridate does still support it. An alternative to removing the feature would be to changing the test not to have trailing zeros in the test data, since those are supported the same. I might suggest that as the quick fix for the test failures.

In terms of ongoing maintenance, maybe you could file an issue on lubridate (or check to see if one exists) and see if this is something they intend to support. We know they plan a major refactor soon, so maybe the maintainers can tell us if this is going to stay or go, and we can drop it or not based on that.

@paleolimbot paleolimbot force-pushed the r-fix-updated-lubridage branch from 4fe9093 to ab67fba Compare November 9, 2022 19:21
@paleolimbot
Copy link
Member Author

Done! We should think about converting the integration tests to actual unit tests, moving them to extra_tests, and running them nightly rather than as part of commit-level CI/tests that run on CRAN. It's good to know when the behaviour diverges but it shouldn't cause every CI check to fail when corner-case behaviour changes elsewhere.

@paleolimbot
Copy link
Member Author

I created ARROW-18285 as a follow-up to revisit this test when this has been addressed in lubridate so that we can merge this (since the test is causing a lot of CI/nightlies to fail).

@paleolimbot paleolimbot merged commit 1e40b0a into apache:master Nov 10, 2022
@paleolimbot paleolimbot deleted the r-fix-updated-lubridage branch November 10, 2022 17:27
@ursabot
Copy link

ursabot commented Nov 11, 2022

Benchmark runs are scheduled for baseline = 6c988db and contender = 1e40b0a. 1e40b0a is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.3% ⬆️0.34%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.18% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 1e40b0ab ec2-t3-xlarge-us-east-2
[Finished] 1e40b0ab test-mac-arm
[Finished] 1e40b0ab ursa-i9-9960x
[Finished] 1e40b0ab ursa-thinkcentre-m75q
[Finished] 6c988db3 ec2-t3-xlarge-us-east-2
[Finished] 6c988db3 test-mac-arm
[Finished] 6c988db3 ursa-i9-9960x
[Finished] 6c988db3 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

kou pushed a commit that referenced this pull request Nov 15, 2022
…4615)

This fixes test failures resulting from an updated lubridate package, whose update removed some functionality that we supported (apparently `lubridate::yq(2021.1)` was a thing). This works in our source because we cast to `string()` before doing any further processing.

Authored-by: Dewey Dunnington <[email protected]>
Signed-off-by: Dewey Dunnington <[email protected]>
kou pushed a commit that referenced this pull request Feb 20, 2023
…Hub issue numbers (#34260)

Rewrite the Jira issue numbers to the GitHub issue numbers, so that the GitHub issue numbers are automatically linked to the issues by pkgdown's auto-linking feature.

Issue numbers have been rewritten based on the following correspondence.
Also, the pkgdown settings have been changed and updated to link to GitHub.

I generated the Changelog page using the `pkgdown::build_news()` function and verified that the links work correctly.

---
ARROW-6338	#5198
ARROW-6364	#5201
ARROW-6323	#5169
ARROW-6278	#5141
ARROW-6360	#5329
ARROW-6533	#5450
ARROW-6348	#5223
ARROW-6337	#5399
ARROW-10850	#9128
ARROW-10624	#9092
ARROW-10386	#8549
ARROW-6994	#23308
ARROW-12774	#10320
ARROW-12670	#10287
ARROW-16828	#13484
ARROW-14989	#13482
ARROW-16977	#13514
ARROW-13404	#10999
ARROW-16887	#13601
ARROW-15906	#13206
ARROW-15280	#13171
ARROW-16144	#13183
ARROW-16511	#13105
ARROW-16085	#13088
ARROW-16715	#13555
ARROW-16268	#13550
ARROW-16700	#13518
ARROW-16807	#13583
ARROW-16871	#13517
ARROW-16415	#13190
ARROW-14821	#12154
ARROW-16439	#13174
ARROW-16394	#13118
ARROW-16516	#13163
ARROW-16395	#13627
ARROW-14848	#12589
ARROW-16407	#13196
ARROW-16653	#13506
ARROW-14575	#13160
ARROW-15271	#13170
ARROW-16703	#13650
ARROW-16444	#13397
ARROW-15016	#13541
ARROW-16776	#13563
ARROW-15622	#13090
ARROW-18131	#14484
ARROW-18305	#14581
ARROW-18285	#14615
* Closes: #33631

Authored-by: SHIMA Tatsuya <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants