fix(resolver): redesign resolver cache

[tiran]

Redesign the resolver cache to address multiple issues.

1. The cache now holds all candidates from the provider. Before it only stored candidates that also fulfilled the requirement and constraints.
2. The cache now takes sdist url, GitHub repo, and GitLab project into account. Before a cache for local PyPI and pypi.org were mixed.
3. GenericProvider no longer caches candidates. There is no way to construct a good cache key. Bootstrapper is using GenericProvider in way that does not benefit from caching either.
4. There is just one global cache object for all providers. The new design makes it easier to clear all caches or just the cache for a single identifier.
5. The custom logic for each provider class is now in find_candidates function. The method just has to return an iterable of candidates. The caching logic is handled by the rest of the code.
6. Consumers can now opt out of caching with use_resolver_cache=False argument.
7. All resolver classes now require keyword arguments. The base provider no longer takes PyPI-only arguments like include_sdists.

Fixes: #766

[tiran]

test_provider_cache is expected to fail for now.

[dhellmann]

It looks like ruff is not happy with the formatting of a file

Would reformat: tests/test_resolver.py

[dhellmann]

I'm happy with this version but I would like another pair of eyes to look at it.

[LalatenduMohanty]

Before (this code):

Cache: {} (empty)
Resolve: numpy<2  → Query PyPI → Cache: {numpy: [1.24, 1.25, 1.26.4]} ✓
Resolve: numpy    → Use cache  → Returns: [1.24, 1.25, 1.26.4] ✗ (missing 2.x!)

After:

Cache: {} (empty)
Resolve: numpy<2  → Query PyPI → Cache: {numpy: [1.24, ..., 1.26.4, 2.0, 2.1, 2.2]} ✓
                  → Filter for <2 → Returns: [1.24, 1.25, 1.26.4] ✓
Resolve: numpy    → Use cache  → Cache: [1.24, ..., 2.2] (all versions)
                  → Filter (no constraint) → Returns: [2.2] ✓ (latest!)

[LalatenduMohanty]

We should add a test demonstrating the numpy scenario from #766
With the code change we need to udpate, docs/hooks.rst for VersionSource Signature

[LalatenduMohanty]

There might be more changes we need to do in the docs/hooks.rst. Cursor identified these cases.

[ ] Fix _version_source() to take only identifier parameter
[ ] Fix return type to Iterable[tuple[str, Version]]
[ ] Update example to return tuples, not just versions
[ ] Add note about keyword-only arguments
[ ] Test the example code actually works

[rd4398]

This looks good. I see @LalatenduMohanty has some comments so I will wait for him to approve

[tiran]

@LalatenduMohanty I have updated and fixed the example in the docs

[LalatenduMohanty]

We should add a test demonstrating the numpy scenario from #766

@tiran I could not find the test covering the numpy scenario. May be you did not add because we do not need it?

[tiran]

@LalatenduMohanty I don't know how to reproduce that problem. Maybe you can come up with a test?

[LalatenduMohanty]

I don't know how to reproduce that problem. Maybe you can come up with a test?

Sounds good. I have a test locally I can send in a separate PR. So lets go ahead and merge this PR.

[LalatenduMohanty]

Created a issue to track this #838