[FEATURE] Support Spark 3.5.1 #1591
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the feature
Since Spark 3.5.1 is released, we should support this version.
Motivation
No response
Describe the solution
No response
Additional context
The abstract class TaskContext introduced a new method isFailed() after Spark version 3.5.1. Due to this addition, it is not possible to compile the MockTaskContext class using Spark 3.5.1 without modifying the code. Consequently, we also added the isFailed() method to MockTaskContext. However, to maintain compatibility with versions of Spark prior to 3.5.1, we did not annotate this method with @Override.
Compiling old code with Spark 3.5.1 results in the following error:
error: MockTaskContext is not abstract and does not override abstract method isFailed() in TaskContext.
Additionally, adding the @Override annotation leads to another error:
error: method does not override or implement a method from a supertype.
Are you willing to submit PR?
- Yes I am willing to submit a PR!