Fix Attention GQA implementation on CPU

[xadupre]

Description

Attention on CPU is following ONNX specifications. This change replicates the changes introduced by onnx/onnx#7274.

[github-actions]

You can commit the suggested changes from lintrunner.

[Copilot]

Pull Request Overview

This PR fixes the Attention GQA (Grouped Query Attention) implementation on CPU to comply with ONNX specifications by updating the test exclusions and correcting the GQA index calculation logic.

Key changes:

• Updated test filters to exclude pending ONNX update tests with appropriate comments
• Fixed GQA head indexing calculation from modulo-based to division-based approach
• Corrected mask value assignment to use negative infinity instead of lowest value

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File	Description
onnxruntime/test/testdata/onnx_backend_test_series_filters.jsonc	Updated test exclusion patterns for attention tests pending ONNX updates
onnxruntime/test/onnx/main.cc	Added hardcoded test exclusions for attention GQA tests pending ONNX updates
onnxruntime/core/providers/cpu/llm/attention.cc	Fixed GQA head indexing calculation and corrected mask values to use negative infinity

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[tianleiwu]

Any change of attention value from changing masked value from lowest() to -inf? I think the result shall be close enough.

[justinchuby]

There is one case where we see NaN from softmax: https://github.com/microsoft/onnxscript/blob/a70ee8d0905f563c840bbd5338595e9ac6b1b5b4/onnxscript/function_libs/torch_lib/ops/nn.py#L2147-L2155

Not sure if it is related

[justinchuby]

I wonder if there are benefits in using -inf? Should we change https://github.com/microsoft/onnxscript/blob/a70ee8d0905f563c840bbd5338595e9ac6b1b5b4/onnxscript/function_libs/torch_lib/ops/nn.py#L2141 to use lowest()?

[xadupre]

The reason why I switch to inifinity is that there is a case where the mask is added to the input. If it is infinity, any value won't change it unless it is infinity itself or nan. That's not true for lowest. I don't think it is likely to happen than the mask is == -lower and the input is +lowest ~highest.

[justinchuby]

Question is do we need special handling for softmax, when mask is -inf?

# When using scaled dot product attention with a boolean mask, the softmax operation might return NaN values
# due to the presence of -inf in an entire row (padding tokens), resulting in 0/0 (NaN) in the softmax output.
# This is because there's no safe/masked softmax imp in ONNX, so we need to handle NaN values explicitly to match
# the behavior of PyTorch with boolean masks.
# Reference: https://github.com/pytorch/pytorch/issues/103749

[xadupre]

Question is do we need special handling for softmax, when mask is -inf?

# When using scaled dot product attention with a boolean mask, the softmax operation might return NaN values
# due to the presence of -inf in an entire row (padding tokens), resulting in 0/0 (NaN) in the softmax output.
# This is because there's no safe/masked softmax imp in ONNX, so we need to handle NaN values explicitly to match
# the behavior of PyTorch with boolean masks.
# Reference: https://github.com/pytorch/pytorch/issues/103749

I see your point. We should add a unit test with such a mask.