SNOW-2173644, SNOW-2157873: Fix __array_function__ hybrid switching and native Series constructor switching bugs #3498
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-2173644 and SNOW-2157873
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
SNOW-2157873 occurs because upstream modin does not implement
__array_function__
, instead converting to numpy ndarrays via__array__
when a numpy function is called on it. The presence of the extension wrapper for__array_function__
introduced by Snowpark pandas confuses numpy dispatch, causing unexpected AttributeErrors. This is fixed upstream with modin-project/modin#7617, and will presumably become available in the next modin release. On the Snowpark side, this PR adds relevant tests, and adds a version-guarded flag to remove the extension function and push it down to the query compiler.SNOW-2173644 occurs in specific circumstances when determining switching conditions for the DataFrame constructor. Series objects are treated as dict-like, but
Series.values
is a property rather than a function. We thus skip over native_pd.Series objects in the dict-like check in move_to_me_cost.