issues with retrieving large image data from GPU #344
Description
Hi @haesleinhuepf
Hope you have been well.
I'm following up on an issue we had with large image data
BioimageAnalysisCoreWEHI/napari_lattice#105
When I run this code
import numpy as np
import pyclesperanto_prototype as cle
np.random.seed(0)
image = np.random.randint(0, 255, (401, 2304, 512), dtype=np.uint16)
angle_in_degrees=35
voxel_size_x=0.101
voxel_size_y=0.101
voxel_size_z=1.2
deskewed = cle.deskew_x(image,angle_in_degrees=35,
voxel_size_x=voxel_size_x,
voxel_size_y=voxel_size_y,
voxel_size_z=voxel_size_z,)
deskewed.shape
(294, 2304, 5184)
The deskew process works.
When I pull the image from the GPU,
deskewed_np = cle.pull_zyx(deskewed)
I get this error
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[2], line 1
----> 1 deskewed_np = cle.pull_zyx(deskewed)
File ~\AppData\Roaming\Python\Python39\site-packages\pyclesperanto_prototype\_tier0\_pull.py:36, in pull_zyx(gpu_array)
31 import warnings
32 warnings.warn(
33 "Deprecated: `pull_zyx()` is now deprecated as it does the same as `pull()`.",
34 DeprecationWarning
35 )
---> 36 return pull(gpu_array)
File ~\AppData\Roaming\Python\Python39\site-packages\pyclesperanto_prototype\_tier0\_pull.py:28, in pull(gpu_array)
4 def pull(gpu_array):
5 """Returns an image from GPU memory as numpy compatible array
6
7 .. deprecated:: 0.6.0
(...)
26 .. [1] https://clij.github.io/clij2-docs/reference_pull
27 """
---> 28 return Backend.get_instance().get().asarray(gpu_array)
File ~\AppData\Roaming\Python\Python39\site-packages\pyclesperanto_prototype\_tier0\_opencl_backend.py:18, in OpenCLBackend.asarray(self, image)
17 def asarray(self, image):
---> 18 return np.asarray(image)
File ~\AppData\Roaming\Python\Python39\site-packages\pyclesperanto_prototype\_tier0\_pycl.py:111, in OCLArray.__array__(self, dtype)
109 def __array__(self, dtype=None):
110 if dtype is None:
--> 111 return self.get()
112 else:
113 return self.get().astype(dtype)
File c:\ProgramData\Miniconda3\envs\napari_lattice_ilastik\lib\site-packages\pyopencl\array.py:921, in Array.get(self, queue, ary, async_, **kwargs)
915 if async_:
916 warn("calling pyopencl.Array.get with 'async_=True' is deprecated. "
917 "Please use pyopencl.Array.get_async for asynchronous "
918 "device-to-host transfers",
919 DeprecationWarning, stacklevel=2)
--> 921 ary, event1 = self._get(queue=queue, ary=ary, async_=async_, **kwargs)
923 return ary
File c:\ProgramData\Miniconda3\envs\napari_lattice_ilastik\lib\site-packages\pyopencl\array.py:880, in Array._get(self, queue, ary, async_, **kwargs)
874 raise ValueError("Cannot copy array to host. "
875 "Array has no queue. Use "
876 "'new_array = array.with_queue(queue)' "
877 "to associate one.")
879 if self.size:
--> 880 event1 = cl.enqueue_copy(queue, ary, self.base_data,
881 src_offset=self.offset,
882 wait_for=self.events, is_blocking=not async_)
884 self.add_event(event1)
885 else:
File c:\ProgramData\Miniconda3\envs\napari_lattice_ilastik\lib\site-packages\pyopencl\__init__.py:2013, in enqueue_copy(queue, dest, src, **kwargs)
2006 warn("The 'device_offset' argument of enqueue_copy "
2007 "is deprecated. Use 'src_offset' instead. "
2008 "'dst_offset' will stop working in 2023.x.",
2009 DeprecationWarning, stacklevel=2)
2011 kwargs["src_offset"] = device_offset
-> 2013 return _cl._enqueue_read_buffer(queue, src, dest, **kwargs)
2015 elif src.type in _IMAGE_MEM_OBJ_TYPES:
2016 origin = kwargs.pop("origin")
RuntimeError: clEnqueueReadBuffer failed: OUT_OF_RESOURCES
It looks like we are running out of memory, but I assumed as the image is already on GPU, it shouldn't be a problem?
Cheers
Pradeep