-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Closed
Labels
Description
Expected Behavior
When defining a field with ValueType.UNIX_TIMESTAMP in a RequestDataSource, the input will be a datetime64[ns] or datetime64[ns, tz] pd.Series.
Current Behavior
ValueType.UNIX_TIMESTAMP is translated to "datetime" which is not understood by the pd.Series constructor. Thus, infer_features fails if there are ValueType.UNIX_TIMESTAMP present in the RequestDataSource schema.
Steps to reproduce
While this will work:
import feast
import pandas as pd
date_request = feast.data_source.RequestDataSource(
name="date_request",
schema={"some_date": feast.ValueType.STRING},
)
@feast.on_demand_feature_view.on_demand_feature_view(
inputs={
"date_request": date_request,
},
features=[
feast.Feature("output", feast.ValueType.STRING),
],
)
def test_view(features_df: pd.DataFrame) -> pd.DataFrame:
data = pd.DataFrame()
data["output"] = features_df["some_date"].astype("category")
return data
test_view.infer_features()
This won't:
import feast
import pandas as pd
date_request = feast.data_source.RequestDataSource(
name="date_request",
schema={"some_date": feast.ValueType.UNIX_TIMESTAMP}, # <-- now a timestamp
)
@feast.on_demand_feature_view.on_demand_feature_view(
inputs={
"date_request": date_request,
},
features=[
feast.Feature("output", feast.ValueType.STRING),
],
)
def test_view(features_df: pd.DataFrame) -> pd.DataFrame:
data = pd.DataFrame()
data["output"] = features_df["some_date"].astype("category")
return data
test_view.infer_features()
Specifications
- Version: 0.15.1
- Platform: macOS
- Subsystem: BigSur
Possible Solution
Change the dtype mapping of ValueType.UNIX_TIMESTAMP from "datetime" to "datetime64[ns]" locally for OnDemandFeatureView.infer_features() or in feast_value_type_to_pandas_type().