Ondemand Feature with STRING type input fails on apply

[OptimeeringBigya]

Expected Behavior

New on demand feature view created.

Current Behavior

Attempting to apply a feature view with STRING type field fails with an exception below.

/home/user/source_code/feast_exp/.venv/bin/python /home/user/source_code/feast_exp/src/od.py 
/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/feature_store.py:580: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
  warnings.warn(
Traceback (most recent call last):
  File "/home/user/source_code/feast_exp/src/od.py", line 23, in <module>
    STORE.apply([r,transform])
  File "/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/feature_store.py", line 936, in apply
    self._make_inferences(
  File "/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/feature_store.py", line 643, in _make_inferences
    odfv.infer_features()
  File "/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/on_demand_feature_view.py", line 654, in infer_features
    inferred_features = self.feature_transformation.infer_features(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/transformation/pandas_transformation.py", line 98, in infer_features
    python_type_to_feast_value_type(
  File "/home/user/source_code/feast_exp/.venv/lib/python3.11/site-packages/feast/type_map.py", line 214, in python_type_to_feast_value_type
    raise ValueError(
ValueError: Value with native type object cannot be converted into Feast value type

Process finished with exit code 1

Steps to reproduce

Attempt to apply the feature view below.

from feast import on_demand_feature_view
from feast import RequestSource
from feast import Field
from feast.types import String,Int32

from feature_store.configs.store import STORE

r=RequestSource(
    name="r",
    timestamp_field="event_ts",
    schema=[
        Field(name="input_1", dtype=String)
    ]
)


@on_demand_feature_view.on_demand_feature_view(name="a",schema=[],entities=[],sources=[r])
def transform(df):
    return df

if __name__ == "__main__":
    STORE.apply([r,transform])

Specifications

• Version: 0.53.0
• Platform: Linux
• Subsystem: Ubuntu

Possible Solution

Running the apply on debug mode shows that the input column is of type object instead of string .
When the column is explicitly type cast to string with the line below in the transform method, the apply successfully passes.
df["input_1"]=df["input_1"].astype("string")

One solution would be to ensure that the input source attempts to type cast to defined type and raise an exception if it cannot be type cast.

[franciscojavierarceo]

did you try following this example:

https://docs.feast.dev/reference/beta-on-demand-feature-view#example-1-on-demand-transformation-on-read-using-pandas-mode

Also, you have an extra on_demand_feature_view after the decorator.

@on_demand_feature_view(
    name="a",
    schema=[],
    entities=[],
    sources=[r]
)
def transform(df):
    return df

[OptimeeringBigya]

did you try following this example:

https://docs.feast.dev/reference/beta-on-demand-feature-view#example-1-on-demand-transformation-on-read-using-pandas-mode

Also, you have an extra on_demand_feature_view after the decorator.

@on_demand_feature_view(
name="a",
schema=[],
entities=[],
sources=[r]
)
def transform(df):
return df

I modified the example from the example link.

from feast import Field, RequestSource, on_demand_feature_view
from feast.types import String
import pandas as pd

# Define a request data source for request-time features
input_request = RequestSource(
    name="vals_to_add",
    schema=[
        Field(name="val_to_add", dtype=String),
        Field(name="val_to_add_2", dtype=String),
    ],
)

# Use input data and feature view features to create new features in Pandas mode
@on_demand_feature_view.on_demand_feature_view(
    sources=[input_request],
    schema=[
        Field(name="conv_rate_plus_val1", dtype=String),
        Field(name="conv_rate_plus_val2", dtype=String),
    ],
    mode="pandas",
)
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
    df = pd.DataFrame()
    df["conv_rate_plus_val1"] = features_df["val_to_add"]
    df["conv_rate_plus_val2"] = features_df["val_to_add_2"]
    return df

if __name__ == "__main__":
    STORE.apply([input_request, transformed_conv_rate])

It still fails with the same error.

Also, the imported on_demand_feature_view in feast python sdk is the module and not the decorator. The example seems to be incorrect with the use of the decorator..

[franciscojavierarceo]

I updated the documentation on that README, that was an error.

[franciscojavierarceo]

@OptimeeringBigya can you try:

@on_demand_feature_view.on_demand_feature_view(
    sources=[input_request],
    schema=[
        Field(name="conv_rate_plus_val1", dtype=String),
        Field(name="conv_rate_plus_val2", dtype=String),
    ],
    mode="pandas",
)
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
    df = pd.DataFrame()
    df["conv_rate_plus_val1"] = features_df["val_to_add"].astype(str)
    df["conv_rate_plus_val2"] = features_df["val_to_add_2"].astype(str)
    return df

[OptimeeringBigya]

@OptimeeringBigya can you try:

@on_demand_feature_view.on_demand_feature_view(
sources=[input_request],
schema=[
Field(name="conv_rate_plus_val1", dtype=String),
Field(name="conv_rate_plus_val2", dtype=String),
],
mode="pandas",
)
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
df = pd.DataFrame()
df["conv_rate_plus_val1"] = features_df["val_to_add"].astype(str)
df["conv_rate_plus_val2"] = features_df["val_to_add_2"].astype(str)
return df

@franciscojavierarceo This is still failing. Replacing .astype(str) with .astype("string") fixes the issue.

[OptimeeringBigya]

#5641 (comment) while this is a hack around the issue. This does not fix the source of the issue.

Example:

# Define a request data source for request-time features
input_request = RequestSource(
    name="vals_to_add",
    schema=[
        Field(name="val_to_add", dtype=String),
    ],
)

# Use input data and feature view features to create new features in Pandas mode
@on_demand_feature_view.on_demand_feature_view(
    sources=[input_request],
    schema=[
        Field(name="hour", dtype=Int32),
        Field(name="mins", dtype=Int32),
    ],
    mode="pandas",
)
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
    """
    This extracts hour and min from the ISO format input string
    """
    features_df["hour"]=3 # assume computed ints
    features_df["mins"]=4 # assume computed ints
    return features_df

if __name__ == "__main__":
    STORE.apply([input_request, transformed_conv_rate])

Applying this still causes an issue, as the input column vals_to_add is of type obj and not string (even though it is defined as string in the input source).

Type converting the input column using .astype(str) feels hackish. I do not think a transformation function should be modifying the input columns and should only be generating the output columns.

[OptimeeringBigya]

Possibly a duplicate of #2229