fix: Silent failure in Curve25519 arithmetic precompiles with malformed points

[librelois]

SRLabs audited the frontier codebase on may 2025: https://dotpal.io/assets/files/frontier-srlabs-2505-718c3bfa5df9fed1862fed05de506859.pdf

Several issues mentioned in the audit are not fixed, this PR aims to fix the issue S2-58: "Silent failure in Curve25519 arithmetic precompiles with malformed points".

Background

The Curve25519 arithmetic precompiles allow smart contract developers to cheaply utilize these
curve operations in their smart contracts.

Issue description

The Curve25519Add and Curve25519ScalarMul precompiles incorrectly handle invalid Ristretto point
representations. Instead of returning an error, they silently treat invalid input bytes as the Ristretto
identity element, leading to potentially incorrect cryptographic results.
In frame/evm/precompile/curve25519/src/lib.rs, when processing input points for both addition
and scalar multiplication, in the execute function, the code attempts to decompress the 32-byte
input using point.decompress().
If the input bytes do not represent a valid compressed Ristretto point, decompress() returns None.
However, the code uses unwrap_or_else(RistrettoPoint::identity), which replaces this None
result with the RistrettoPoint::identity() element without signalling any error to the caller.

Risk

Silently treating invalid cryptographic points as the identity element could introduce critical
vulnerabilities in smart contracts. This behavior could compromise confidentiality in key exchanges
or allow threshold signature requirements to be bypassed. For example, in a multi-signature scheme,
an attacker could submit an invalid compressed Ristretto point as a public key. Treated as the
identity element, it would be incorrectly counted toward the signing threshold, enabling signature
forgery with fewer valid participants.

Mitigation

We recommend modifying the execute function in both Curve25519Add and Curve25519ScalarMul.
Instead of using unwrap_or_else(RistrettoPoint::identity), check the result
of point.decompress(). If it returns None, the function should immediately return
a PrecompileFailure::Error indicating invalid point data, rather than proceeding with the identity
element.

[sorpaas]

Security advisory: GHSA-v4q3-23rh-w5mw