GeGLU kernel numerical issue #959
Description
🐛 Describe the bug
The current Liger geglu kernel is not close to torch's implementation, even in fp32. test_geglu.py can't pass with lower tolerances.
Some models couldn't pass the convergence test with low tolerance due to this issue. We need to explore whether there is a better implementation that is closer to torch's.
Reproduce
No response
Versions
Liger Kernel version: 0.6.4
PyTorch version: 2.7.1+cu126