Skip to content

Bug: Snowflake online store writes are not committed #5431

@j-wine

Description

@j-wine

Expected Behavior

Data written using Feast's SnowflakeOnlineStore.online_write_batch should be persisted in the Snowflake table and be visible in subsequent reads, even across different connections.

Current Behavior

Data written by Feast is not persisted. It is sometimes visible in the same session (temporary reads succeed), but disappears after the connection is closed.
In Snowflake UI the table shows 0B, and subsequent reads return empty results.

Steps to Reproduce

  1. Use a StreamFeatureView, which triggers the creation of a Table for that featureview.
  2. Use Feast (with SnowflakeOnlineStore) and Spark-Kafka Processor to write features via online_write_batch.
  3. Immediately read back data with online_read → values are present.
  4. After the connection/session is closed:
  • The table shows 0B in Snowflake UI.
  • Reads return no data (entity ids missing).

Specifications

  • Version: Feast main branch / latest (please update with your Feast version)
  • Platform: Snowflake
  • Subsystem: SnowflakeOnlineStore

Possible Solution

The issue is caused by a missing conn.commit() in online_write_batch.
In SnowflakeOnlineStore.online_write_batch, Feast uses:

with GetSnowflakeConnection(config.online_store, autocommit=False) as conn:
write_pandas_binary(...)
execute_snowflake_statement(conn, query) # INSERT OVERWRITE
# Missing: conn.commit()

Since autocommit=False, the transaction is not committed - and changes are discarded when the connection is closed.
Fix: Add an explicit conn.commit():

with GetSnowflakeConnection(config.online_store, autocommit=False) as conn:
write_pandas_binary(...)
execute_snowflake_statement(conn, query)
conn.commit() # required for persistence

Why This Happens

  • Tests arent failing as during the same session, uncommitted data is visible (why initial reads worked) => integreation tests with new connection would be helpful.
  • Once the connection closes, uncommitted transaction is discarded (why the table is empty and reads fail later)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions