能否将更多的操作移动到CPU?

[jeffrey4l]

我用2张 t4 的显卡跑 deepseek r1 , 但是显存总是不够，还是差一点，有没有可能调下 DeepSeek-V3-Chat-multi-gpu.yaml 文件，把部分 layer 也放到 cpu 上面运行。自己尝试改过，但报下面的错误

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/kt-env2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ktransformers/ktransformers/models/modeling_deepseek_v3.py", line 1202, in forward
    hidden_states = self.input_layernorm(hidden_states)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/kt-env2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/kt-env2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ktransformers/ktransformers/models/modeling_deepseek_v3.py", line 108, in forward
    return self.weight * hidden_states.to(input_dtype)
           ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!