[python3] Expose paths of included assets such as the standard library

[ImOnALampshade]

Is your feature request related to a problem? Please describe.

Python3_STDLIB is exposed after find_package(Python3 COMPONENTS Interpreter Development REQUIRED) in cmake, but isn't neccisarily the python standard library you should configure embedded python to use. On my Linux build, I have to use ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/debug/lib/python${Python3_VERSION_MAJOR}.${Python3_VERSION_MINOR} on debug builds, but Python3_STDLIB always points toward ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/lib/python${Python3_VERSION_MAJOR}.${Python3_VERSION_MINOR}.

On Windows, it's even worse - Python3_STDLIB points to ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/tools/python${Python3_VERSION_MAJOR}/Lib, but there is nothing to point at ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/tools/python${Python3_VERSION_MAJOR}/DLLs, so I have to manually create point toward that.

All this leads to a bunch of CMake code I have in my own build system to set up Python3_STDLIB, Python3_DYNAMIC_LIBARIES for each triplet. Which I was under the impression is VCPKG's job.

Proposed solution

Expose Python3_STDLIB correctly (Either by pointing to the debug or non-debug version of the python libs depending on the cmake configuration, or creating a Python3_STDLIB_DEBUG variable if that isn't possible for whatever reason).

Expose Python3_DYNAMIC_LIBARIES as its own variable on systems where these are not part of the Python3_STDLIB directory (namely; windows).

If these variables are already exposed and I just missed them, it would be great to see them mentioned in the usage file for Python3, which they currently are not.

Describe alternatives you've considered

Currently I am doing the above proposed solution manually in my own build system, but it's fragile - it depends on VCPKG placing files in particular locations,

Additional context

Apps including embedded Python will require the python built-in libraries to be included with the application's distribution. Having to wrangle these all manually is a hassle for getting automated CI builds to work properly.

[Osyotr]

This should be reported as a CMake bug. Vcpkg shouldn't make up new variables.

[ImOnALampshade]

I'll file a bug on them for not including a Python3_DYNAMIC_LIBARIES variable in FindPython3 (here)- however, providing a path to the debug version of Python3_STDLIB on Linux is definitely a VCPKG issue and not related to cmake.

[Osyotr]

How should vcpkg (and CMake) know which interpreter do you mean when you reference Python3_STDLIB (and Python3_DYNAMIC_LIBARIES)? Should both debug and release interpreters point to the same dir?

[ImOnALampshade]

Python3_STDLIB does not refer to the python interpreter. It refers to the Python standard library.

There's a difference here between what VCPKG does on Windows and on Linux, so the answer to your question is different on both systems.

On Windows, there's only one version of the standard library, that gets put in ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/tools/python3/Lib. Python3_STDLIB should link only to that directory - which it does. No problem here.

On Linux, there's two versions of the standard library. The debug version gets placed in ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/debug/lib/python3.12/, while the release version get put in ${VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/lib/python3.12. So depending on if you're building a debug or a release package, the correct library needs to be copied over to the package. HOWEVER - Python3_STDLIB only points to the release version of the standard library. So, this should be changed to point to the debug version for debug builds.

[Osyotr]

Vcpkg could try to help by doing something like this

vcpkg/scripts/buildsystems/vcpkg.cmake

Line 427 in e140b1f

if(NOT DEFINED CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE MATCHES "^[Dd][Ee][Bb][Uu][Gg]$")

for Python3_STDLIB but it is of course only for single-config generators and might be surprising for people who expect Python3_STDLIB to point to the same dir as their release interpreter in tools/.
My observations so far:

1. Python3_STDLIB is set based on what Python interpreter returns. Currently, vcpkg builds only release interpreter. Stdlib is copied to tools.
2. On Windows, standard library does not depend on the type of interpreter. Python3_STDLIB always points to the correct dir.
3. On Windows, official Python installation places debug and release extension modules into the same directory, so I'm not sure what purpose would imaginary Python3_DYNAMIC_LIBARIES have outside of vcpkg's very special installation layout which wouldn't do that (because there would be name conflicts).
4. On Linux, stdlib is per-confif and extension modules are located inside it, in lib-dynload subdir. There It would make sense to have Python3_STDLIB and Python3_DYNAMIC_LIBARIES, but only with DEBUG/RELEASE suffixes.
5. On both Windows and Linux vcpkg should build debug interpreter.