-
Notifications
You must be signed in to change notification settings - Fork 6.9k
[Data] Compute Expressions-datetime #58740
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces datetime expression operations under the .dt namespace, which is a great addition. The implementation in dt_namespace.py is mostly clean and follows existing patterns.
However, there is a critical omission: the dt property is not defined on the Expr class in python/ray/data/expressions.py. Without this, expressions like col('my_date').dt.year() will fail. This property needs to be added, similar to how .list, .str, and .struct are implemented. You can add the following property to the Expr class:
@property
def dt(self) -> "_DatetimeNamespace":
"""Access datetime operations for this expression."""
from ray.data.namespace_expressions.dt_namespace import _DatetimeNamespace
return _DatetimeNamespace(self)I've also left a couple of comments in dt_namespace.py regarding potential improvements:
- A high-severity issue with how the return data type is determined for
ceil,floor, androundfunctions, which could lead to incorrect type inference. - A medium-severity suggestion to refactor duplicated code for better maintainability.
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
86ce74f to
5a04f43
Compare
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Signed-off-by: 400Ping <[email protected]>
Description
Completing the datetime namespace operations
Related issues
Related to #58674
Additional information