Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Clickhouse offline store #4725

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

iamhatesz
Copy link

What this PR does / why we need it:

This PR adds a new contrib offline store backed by Clickhouse.

Which issue(s) this PR fixes:

Lack of Clickhouse support :)

Misc

The implementation is heavily based on the Postgres store and tested against it. The resulting features were identical to the point that it's possible with two different backends (e.g., different data types).

I added a helper to run integration tests: make test-python-universal-clickhouse-offline. Unfortunately, 3 test cases are failing:

ERROR sdk/python/tests/integration/registration/test_universal_types.py::test_feature_get_historical_features_types_match[TypeTestConfig(feature_dtype='float', feature_is_list=True, has_empty_list=False)-ParameterSet(values=(LOCAL:Clickhouse:RedisOnlineStoreCreator:python_fs:False,), marks=[], id=None)] - TypeError: object of type 'NoneType' has no len()
ERROR sdk/python/tests/integration/registration/test_universal_types.py::test_feature_get_historical_features_types_match[TypeTestConfig(feature_dtype='bool', feature_is_list=True, has_empty_list=False)-ParameterSet(values=(LOCAL:Clickhouse:RedisOnlineStoreCreator:python_fs:False,), marks=[], id=None)] - TypeError: object of type 'NoneType' has no len()
ERROR sdk/python/tests/integration/registration/test_universal_types.py::test_feature_get_historical_features_types_match[TypeTestConfig(feature_dtype='datetime', feature_is_list=True, has_empty_list=False)-ParameterSet(values=(LOCAL:Clickhouse:RedisOnlineStoreCreator:python_fs:False,), marks=[], id=None)] - TypeError: object of type 'NoneType' has no len()

This is because Clickhouse doesn't support Nullable(Array(...)) type. I could have added test_universal_types to the ignore list, but I thought it's worth keeping it on, as many other test cases from this test are passing.

Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
@iamhatesz iamhatesz requested a review from a team as a code owner October 31, 2024 13:31
Signed-off-by: Tomasz Wrona <tomasz@cast.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant