Skip to content

fix: disabled data deletion when feature view is deleted #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 3, 2023
Merged

fix: disabled data deletion when feature view is deleted #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5516a08
disabled data deletion when feature view is deleted
Aug 2, 2023
01fd9fc
fixed formatting issues
Aug 2, 2023
c16642a
mysql version latest updated to 8.1.0
Aug 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions sdk/python/feast/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
from feast.infra.offline_stores.contrib.athena_offline_store.athena_source import (
AthenaSource,
)
from feast.infra.offline_stores.contrib.spark_offline_store.spark_source import (
SparkSource,
)
from feast.infra.offline_stores.file_source import FileSource
from feast.infra.offline_stores.redshift_source import RedshiftSource
from feast.infra.offline_stores.snowflake_source import SnowflakeSource
Expand Down Expand Up @@ -54,4 +57,5 @@
"PushSource",
"RequestSource",
"AthenaSource",
"SparkSource",
]
2 changes: 1 addition & 1 deletion sdk/python/feast/feature_view.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
from feast.usage import log_exceptions
from feast.value_type import ValueType

warnings.simplefilter("once", DeprecationWarning)
warnings.simplefilter("ignore", DeprecationWarning)

# DUMMY_ENTITY is a placeholder entity used in entityless FeatureViews
DUMMY_ENTITY_ID = "__dummy_id"
Expand Down
21 changes: 12 additions & 9 deletions sdk/python/feast/infra/aws.py
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we disabling all kind of updates or does this function do delete alone?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this is only called during the Apply phase. Not use anywhere else.

Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,19 @@ def update_infra(
entities_to_keep: Sequence[Entity],
partial: bool,
):
# TODO: Temporarily disabling the capability of Deleting data
# from online stores when a Feature View is deleted.

# Call update only if there is an online store
if self.online_store:
self.online_store.update(
config=self.repo_config,
tables_to_delete=tables_to_delete,
tables_to_keep=tables_to_keep,
entities_to_keep=entities_to_keep,
entities_to_delete=entities_to_delete,
partial=partial,
)
# if self.online_store:
# self.online_store.update(
# config=self.repo_config,
# tables_to_delete=tables_to_delete,
# tables_to_keep=tables_to_keep,
# entities_to_keep=entities_to_keep,
# entities_to_delete=entities_to_delete,
# partial=partial,
# )

if self.repo_config.feature_server and self.repo_config.feature_server.enabled:
warnings.warn(
Expand Down
4 changes: 2 additions & 2 deletions sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@
from feast.type_map import spark_schema_to_np_dtypes
from feast.usage import log_exceptions_and_usage

# Make sure spark warning doesn't raise more than once.
warnings.simplefilter("once", RuntimeWarning)
# Make sure spark warning are ignored
warnings.simplefilter("ignore", RuntimeWarning)


class SparkOfflineStoreConfig(FeastConfigBaseModel):
Expand Down
3 changes: 3 additions & 0 deletions sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@

logger = logging.getLogger(__name__)

# Make sure spark warning are ignored
warnings.simplefilter("ignore", RuntimeWarning)


class SparkSourceFormat(Enum):
csv = "csv"
Expand Down
4 changes: 2 additions & 2 deletions sdk/python/tests/unit/test_sql_registry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,8 @@ def mysql_registry():

container.start()

# The log string uses '8.0.*' since the version might be changed as new Docker images are pushed.
log_string_to_wait_for = "/usr/sbin/mysqld: ready for connections. Version: '8.0.*' socket: '/var/run/mysqld/mysqld.sock' port: 3306"
# The log string uses '8.1.*' since the version might be changed as new Docker images are pushed.
log_string_to_wait_for = "/usr/sbin/mysqld: ready for connections. Version: '8.1.*' socket: '/var/run/mysqld/mysqld.sock' port: 3306"
waited = wait_for_logs(
container=container,
predicate=log_string_to_wait_for,
Expand Down