Skip to content

ODFV does not allow feast.ValueType.UNIX_TIMESTAMP in RequestDataSource schema #2053

@ArrichM

Description

@ArrichM

Expected Behavior

When defining a field with ValueType.UNIX_TIMESTAMP in a RequestDataSource, the input will be a datetime64[ns] or datetime64[ns, tz] pd.Series.

Current Behavior

ValueType.UNIX_TIMESTAMP is translated to "datetime" which is not understood by the pd.Series constructor. Thus, infer_features fails if there are ValueType.UNIX_TIMESTAMP present in the RequestDataSource schema.

Steps to reproduce

While this will work:

import feast
import pandas as pd

date_request = feast.data_source.RequestDataSource(
    name="date_request",
    schema={"some_date": feast.ValueType.STRING},
)
@feast.on_demand_feature_view.on_demand_feature_view(
    inputs={
        "date_request": date_request,
    },
    features=[
        feast.Feature("output", feast.ValueType.STRING),
    ],
)
def test_view(features_df: pd.DataFrame) -> pd.DataFrame:
    data = pd.DataFrame()
    data["output"] = features_df["some_date"].astype("category")
    return data

test_view.infer_features()

This won't:

import feast
import pandas as pd

date_request = feast.data_source.RequestDataSource(
    name="date_request",
    schema={"some_date": feast.ValueType.UNIX_TIMESTAMP},  # <-- now a timestamp
)
@feast.on_demand_feature_view.on_demand_feature_view(
    inputs={
        "date_request": date_request,
    },
    features=[
        feast.Feature("output", feast.ValueType.STRING),
    ],
)
def test_view(features_df: pd.DataFrame) -> pd.DataFrame:
    data = pd.DataFrame()
    data["output"] = features_df["some_date"].astype("category")
    return data

test_view.infer_features()

Specifications

  • Version: 0.15.1
  • Platform: macOS
  • Subsystem: BigSur

Possible Solution

Change the dtype mapping of ValueType.UNIX_TIMESTAMP from "datetime" to "datetime64[ns]" locally for OnDemandFeatureView.infer_features() or in feast_value_type_to_pandas_type().

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions