CB: Hetero pipeline parallel support

[WeldonWangwang]

depend on: openvinotoolkit/openvino#30371

Tickets:CVS-164805

[WeldonWangwang]

Hi @Wovchena , I check the failed test case in CI with openvino-genai 2025.2.0.0 locally, some cases failed too, for example:
test_cb_streamer_vs_return_vs_stateful in https://github.com/openvinotoolkit/openvino.genai/actions/runs/15752272348/job/44400676499?pr=2227

Can you help to check if the CI works well? thanks!!!

[Wovchena]

CI is broken. I don't know the component to blame yet

[WeldonWangwang]

Hi @Wovchena , i re-run the failed item in merge queue, but it seems can not be merged again after failed.

[Copilot]

Pull Request Overview

This PR extends continuous batching to support multi-GPU execution by updating device assertions and block sizing logic.

• Relax the assertion to allow single CPU, single GPU, or multiple GPUs.
• Introduce all_gpu_device to drive block size and context initialization.
• Replace per-GPU flag with all_gpu_device checks in cache manager.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/cpp/src/continuous_batching/pipeline_impl.cpp	Relax device assertion for heterogeneous pipelines and detect all-GPU deployments.
src/cpp/src/continuous_batching/cache_manager.hpp	Use all_gpu_device for block size selection and context setup, replacing is_gpu logic.

Comments suppressed due to low confidence (1)

src/cpp/src/continuous_batching/pipeline_impl.cpp:107

• [nitpick] Consider renaming all_gpu_device to all_gpu_devices to better reflect that it checks a collection of devices.

    const bool all_gpu_device =

[Copilot]

The assertion allows empty execution_devices (since all_of on an empty vector is true). Add a check to ensure execution_devices is non-empty before accessing index 0.

[Copilot]

As above, all_gpu_device will be true for an empty vector. Ensure execution_devices is not empty before using element 0 or combine this into the assertion.