Skip to content

Conversation

ntkathole
Copy link
Member

@ntkathole ntkathole commented Jun 6, 2025

What this PR does / why we need it:

This PR adds support for numpy>2.

Key points:

  • 2.2.2 version of PyTorch works only with NumPy 1.x. So, it is important to upgrade torch>2.2.2.
  • Pytorch macos x86 builds are deprecated, so we have to remove macos-13 and use macos-14 instead in our CI.
  • PyPI has CUDA-enabled PyTorch wheels as the primary distribution. Since our linux github runners need cpu-only build, we are installing torch from index https://download.pytorch.org/whl/cpu.
  • pyspark needs to be updated as 'np.NaN` was removed in the NumPy 2.0 release.

Which issue(s) this PR fixes:

#5074

Misc

  • MacOS x86_64 builds are no longer supported by official PyTorch wheels.

If you're using an Intel-based Mac (not Apple Silicon) or running Python with Rosetta-translated x86_64 architecture:

You cannot install PyTorch versions 2.3.0+ via pip, as no compatible wheels are published. The last supported version is: torch==2.2.2.

To upgrade to newer versions like torch==2.7.0, you must use macOS 14+ on Apple Silicon (arm64) and run a matching arm64 Python environment.

To check your architecture:
python3 -c "import platform; print(platform.machine())"
If the output is x86_64, you’ll be limited to PyTorch ≤ 2.2.2.
If it’s arm64, you can install the latest PyTorch.

@ntkathole ntkathole requested a review from a team as a code owner June 6, 2025 05:55
@ntkathole ntkathole changed the title chore: update numpy chore: Update numpy Jun 6, 2025
@ntkathole ntkathole force-pushed the upgrade_numpy branch 4 times, most recently from 70fec5e to dfe82c9 Compare June 6, 2025 06:51
@ntkathole ntkathole self-assigned this Jun 6, 2025
Signed-off-by: ntkathole <[email protected]>
Copy link
Member

@franciscojavierarceo franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit

FEAST_IS_LOCAL_TEST=True \
FEAST_LOCAL_ONLINE_CONTAINER=True \
python -m pytest --tb=short -v -n 8 --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \
python -m pytest --tb=short -v -n 4 --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 4?

Copy link
Member Author

@ntkathole ntkathole Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently with 8, runner is intermittently throwing Error: The operation was canceled. (not related to this PR, happening on all PRs) due to high memory load on node. For stability, reduced to 4.

@franciscojavierarceo franciscojavierarceo merged commit eb0e799 into feast-dev:master Jun 7, 2025
16 of 17 checks passed
j-wine pushed a commit to j-wine/feast that referenced this pull request Jun 7, 2025
* chore: update numpy

Signed-off-by: ntkathole <[email protected]>

* chore: Reduce concurrency to save memory on runners

Signed-off-by: ntkathole <[email protected]>

---------

Signed-off-by: ntkathole <[email protected]>
Signed-off-by: Jacob Weinhold <[email protected]>
devin-ai-integration bot pushed a commit to franciscojavierarceo/feast that referenced this pull request Jun 9, 2025
* chore: update numpy

Signed-off-by: ntkathole <[email protected]>

* chore: Reduce concurrency to save memory on runners

Signed-off-by: ntkathole <[email protected]>

---------

Signed-off-by: ntkathole <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants