Skip to content

Conversation

@lidavidm
Copy link
Member

No description provided.

@github-actions
Copy link

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much of this will need to be reconciled with ARROW-13390/#11080.

@pitrou
Copy link
Member

pitrou commented Sep 29, 2021

This needs rebasing now.

@lidavidm lidavidm force-pushed the arrow-13358 branch 2 times, most recently from bca16d4 to 870c1a6 Compare September 29, 2021 18:00
@lidavidm
Copy link
Member Author

Contains some changes from #11272 so if_else/case_when/coalesce have mostly similar implicit cast rules (though I will leave it to ARROW-14105 to fully unify these). I may also revisit the dictionary case here as it seems slow from test runtime; instead of unifying the dictionaries on-the-fly we should unify them up front and construct a transposition map. (If we copy the RHS to the output, we can use the existing function for transposing indices, and assuming unification preserves the first dictionary's indices exactly, we can then copy the LHS to the output without having to transpose again.)

@lidavidm
Copy link
Member Author

Ah, the described optimization would require being able to unify dictionaries with nulls. Filed ARROW-14177 for now.

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. +1 from me.

@pitrou pitrou closed this in dfb109c Oct 4, 2021
ViniciusSouzaRoque pushed a commit to s1mbi0se/arrow that referenced this pull request Oct 20, 2021
Closes apache#11218 from lidavidm/arrow-13358

Lead-authored-by: David Li <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants