Could not import module 'feast.infra.offline_stores.snowflake'

[wjhhuizi]

Expected Behavior

Feast should correctly import 'feast.infra.online_stores.snowflake' or 'feast.infra.offline_stores.snowflake' modules when being invoked in CLI or SDK.

Current Behavior

When running any command with FEAST CLI, if in feature_store.yaml we provide

offline_store:
    type: snowflake.offline

as our offline storage, the console would return following error stack trace and exit:

Feast is an open source project that collects anonymized error reporting and usage statistics. To opt out or learn more see https://docs.feast.dev/reference/usage
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/feast/importer.py", line 26, in import_class
    module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/feast/infra/online_stores/snowflake.py", line 15, in <module>
    from feast.infra.utils.snowflake.snowflake_utils import (
  File "/usr/local/lib/python3.8/site-packages/feast/infra/utils/snowflake/snowflake_utils.py", line 28, in <module>
    import snowflake.connector
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/__init__.py", line 16, in <module>
    from .connection import SnowflakeConnection
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/connection.py", line 25, in <module>
    from . import errors, proxy
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/errors.py", line 18, in <module>
    from .telemetry_oob import TelemetryService
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/telemetry_oob.py", line 20, in <module>
    from .vendored import requests
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/requests/__init__.py", line 119, in <module>
    from ..urllib3.contrib import pyopenssl
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.SSL
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/crypto.py", line 3268, in <module>
    _lib.OpenSSL_add_all_algorithms()
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/feast", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/cli.py", line 178, in ui
    store = FeatureStore(repo_path=str(repo), fs_yaml_file=fs_yaml_file)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 362, in wrapper
    raise exc.with_traceback(traceback)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 348, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/feature_store.py", line 152, in __init__
    self.config = load_repo_config(self.repo_path, fs_yaml_file)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 548, in load_repo_config
    c = RepoConfig(**raw_config)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 184, in __init__
    super().__init__(**data)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1050, in pydantic.main.validate_model
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 362, in wrapper
    raise exc.with_traceback(traceback)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 348, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 346, in _validate_online_store_config
    online_config_class = get_online_config_from_type(online_store_type)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 518, in get_online_config_from_type
    return import_class(module_name, config_class_name, config_class_name)
  File "/usr/local/lib/python3.8/site-packages/feast/importer.py", line 31, in import_class
    raise FeastModuleImportError(module_name, class_name) from e
feast.errors.FeastModuleImportError: Could not import module 'feast.infra.online_stores.snowflake' while attempting to load class 'SnowflakeOnlineStoreConfig'

This issue is currently existing in version 0.27.0 all the way up to 0.30.2.

There is a relevant issue (#2560) which marked as closed but the fix was not effective.

Steps to reproduce

Provide the feature_store.yaml and run any feast cli command with version of 0.27.0 or above

project: ProjectName

registry:
    registry_type: sql
    path: mysql://XXXX:@hostname:3306/mysql
    cache_ttl_seconds: 600

provider: local

online_store:
    type: sqlite
    path: data/online_store.db

offline_store:
    type: snowflake.offline
    account: xxxxxx.region.privatelink
    user: XXX_XXX_XXX
    password: 'P@SSWORD123$'
    role: 'XXX_XXX_XXXXXX_XXX'
    warehouse: XX_XX_XXXX_XXXX
    database: 'XXXX_XX_XXXXX_XX'
    schema: 'XXXX_XXX_XXXXXX_XX'
    
entity_key_serialization_version: 2

IF no online_store provided, the error will then reflecting as Could not import module 'feast.infra.online_stores.snowflake'

Specifications

• Version: 0.27.0, 0.28.0, 0.29.0, 0.30.2
• Platform: Linux
• Subsystem: Linux

Possible Solution

[wjhhuizi]

AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

This could be the direct cause of the issue.

[Kavee30]

Hi @achals
Need your input in fixing this bug.

[wjhhuizi]

After did some research and did pip3 install cryptography==38.0.4, another error raised

feast.errors.FeastClassImportError: Could not import class 'SqlRegistryConfig' from module 'feast.infra.registry.sql'

[sfc-gh-madkins]

@wjhhuizi does it work with a different registry than SQL?

[sfc-gh-madkins]

I think #3587 should solve this

[davidschuler-8451]

After did some research and did pip3 install cryptography==38.0.4, another error raised

feast.errors.FeastClassImportError: Could not import class 'SqlRegistryConfig' from module 'feast.infra.registry.sql'

this is the same issue as #3544. I just opened a PR to address that but need some feedback on if it's the right approach or not #3586

[Kavee30]

@davidschuler-8451 @sfc-gh-madkins With feast version - 0.30.2, even the command 'feast init -t snowflake' throws the same error

[wjhhuizi]

It seems the PR #3587 was closed, what will happen next? @sfc-gh-madkins

[sfc-gh-madkins]

I cant confirm that PR would have solved the issued. I need more details than the error you are providing.

…

On Wed, Apr 12, 2023 at 2:27 PM Michael Wu ***@***.***> wrote: It seems the PR #3587 <#3587> was closed, what will happen next? @sfc-gh-madkins <https://github.com/sfc-gh-madkins> — Reply to this email directly, view it on GitHub <#3580 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATSRCU4VIYQVSBODFYWANDDXA362TANCNFSM6AAAAAAWO7X4MY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wjhhuizi]

Please let me know what type of detail do you need, let me see if I can help to provide :-)