Handling of different packages with same name (Pillow and Pillow-SIMD) is confusing.

[MalteEbner]

Summary

The way how uv handles different packages with the same name is very confusing. I tried it with pillow-simd and pillow.

The order in which pillow-simd and pillow are installed can cause the wrong version of them to be used or the installation to be missing. Basically there are 5 different states:

Pillow-SIMD	Pillow	Notes
Uninstalled	Uninstalled	Neither package is present
Installed & Active	Uninstalled	Only Pillow-SIMD is installed and used
Uninstalled	Installed & Active	Only Pillow is installed and used
Installed & Active	Installed & Inactive	Both installed; Pillow-SIMD is the active one
Installed & Inactive	Installed & Active	Both installed; Pillow is the active one

The problem is now that I cannot define that never both should be installed, but only one. I tried many different approaches, but non of them worked.

E.g. I used dependency groups, (one with pillow and one with pillow-simd). However, due to main or dev dependencies having pillow as transitive dependency, there still was the problem of both being installed and the active one depending on the order of installation or the order of dependency groups.

Wished behaviour.

I would like to tell that always either pillow or pillow-simd is installed, but never both. How can I do that?

Reproduction examples.

Two examples for weird behaviours:

No fallback to pillow

Let's start with a completely fresh pyproject.toml and .venv

ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ rm -R .venv/
ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv venv .venv --python 3.10
Using CPython 3.10.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ source .venv/bin/activate
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'PIL'

# Let's add pillow for arm64 and pillow-simd for amd64. The versions are just as expected.
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv add pillow --group arm64
Resolved 2 packages in 26ms
Installed 1 package in 2ms
 + pillow==11.2.1
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
11.2.1
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv add pillow-simd --group amd64
Resolved 3 packages in 22ms
Installed 1 package in 6ms
 + pillow-simd==9.5.0.post2
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
9.5.0.post2

# Let's add matplotlib, which requires pillow
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv add matplotlib
Resolved 14 packages in 40ms
Installed 10 packages in 14ms
 + contourpy==1.3.2
 + cycler==0.12.1
 + fonttools==4.58.3
 + kiwisolver==1.4.8
 + matplotlib==3.10.3
 + numpy==2.2.6
 + packaging==25.0
 + pyparsing==3.2.3
 + python-dateutil==2.9.0.post0
 + six==1.17.0
# It still uses pillow-simd
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
9.5.0.post2
# Let's sync again, without the amd64 group
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv sync
Resolved 14 packages in 9ms
Uninstalled 1 package in 1ms
 - pillow-simd==9.5.0.post2
 
# Error: pillow is not installed even though it is required by matplotlib
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name 'Image' from 'PIL' (unknown location)

Not using pillow-simd even though it is defined in a group

Let's start with a completely fresh pyproject.toml and .venv

ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv venv .venv --python 3.10
Using CPython 3.10.13
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ source .venv/bin/activate
# Let's install pillow-simd in a group
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv add pillow-simd --group amd64
Resolved 2 packages in 13ms
Installed 1 package in 2ms
 + pillow-simd==9.5.0.post2
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
9.5.0.post2
# Let's add matplotlib. It now installs pillow instead of using pillow-simd
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv add matplotlib
Resolved 14 packages in 25ms
Installed 11 packages in 14ms
 + contourpy==1.3.2
 + cycler==0.12.1
 + fonttools==4.58.3
 + kiwisolver==1.4.8
 + matplotlib==3.10.3
 + numpy==2.2.6
 + packaging==25.0
 + pillow==11.2.1
 + pyparsing==3.2.3
 + python-dateutil==2.9.0.post0
 + six==1.17.0
# As expected, pillow (non-SIMD) is now used
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
11.2.1
# Let's install the amd64 group with pillow-simd again. Nothing happens
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv sync --group amd64
Resolved 14 packages in 9ms
Audited 12 packages in 0.01ms
# Still using pillow (non-SIMD)
(.venv) ubuntu@ip-172-31-20-46:/GitHub/lightly-core/test_pillow$ uv run python -c "from PIL import Image; print(Image.__version__)"
11.2.1

Platform

Ubuntu 20.04 amd64

Version

0.7.5

Python version

3.10.13

[konstin]

I think that's another case of #13437: The two packages contain the same modules, so installing them together leads to a race condition on installation. Or in your case, one package overrides the other and then also removes those files. Generally, uv can't tell that pillow-simd is also pillow and can be used with matplotlib, I don't think they can be used together without hacks.

[MalteEbner]

Suggested solution: I think it should be possible for the user to define different packages under the same module name similar to the sources logic. Something similar to this:

[tool.uv.sources]
pillow = [
  { package = "pillow-simd>=9,<10", marker = "platform_machine == 'x86_64'"},
  { package = "pillow>=11,<12", marker = "platform_machine == 'aarch64'"},
]

[konstin]

This works, but the problem is that matplotlib depends on pillow, not on pillow-simd, and there's no mechanism in python packaging to tell that pillow-simd is a replacement for pillow, so you can't mix pillow-simd and matplotlib.

[zanieb]

This is tracked in #4422