Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add docs for Go feature server #2540

Merged
merged 8 commits into from
Apr 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
* [.feastignore](reference/feature-repository/feast-ignore.md)
* [Feature servers](reference/feature-servers/README.md)
* [Local feature server](reference/feature-servers/local-feature-server.md)
* [Go-based feature retrieval](reference/feature-servers/go-feature-retrieval.md)
* [\[Alpha\] Data quality monitoring](reference/dqm.md)
* [\[Alpha\] On demand feature view](reference/alpha-on-demand-feature-view.md)
* [\[Alpha\] Stream ingestion](reference/alpha-stream-ingestion.md)
Expand Down
38 changes: 38 additions & 0 deletions docs/reference/feature-servers/go-feature-retrieval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Go-based Feature Retrieval

## Overview

The Go Feature Retrieval component is a Go implementation of the core feature serving logic, embedded in the Python SDK. It supports retrieval of feature references, feature services, and on demand feature views, and can be used either through the Python SDK or the [Python feature server](local-feature-server.md).

Currently, this component only supports online serving and does not have an offline component including APIs to create feast feature repositories or apply configuration to the registry to facilitate online materialization. It also does not expose its own dedicated cli to perform feast actions. Furthermore, this component is only meant to expose an online serving API that can be called through the python SDK to facilitate faster online feature retrieval.

The Go Feature Retrieval component currently only supports Redis and Sqlite as online stores; support for other online stores will be added soon. Initial benchmarks indicate that it is significantly faster than the Python feature server for online feature retrieval. We plan to release a more comprehensive set of benchmarks. For more details, see the [RFC](https://docs.google.com/document/d/1Lgqv6eWYFJgQ7LA_jNeTh8NzOPhqI9kGTeyESRpNHnE).

## Installation

As long as you are running macOS or linux x86 with python version 3.7-3.10, the go component comes pre-compiled when you run install feast.
kevjumba marked this conversation as resolved.
Show resolved Hide resolved

For developers, if you want to build from source, run `make compile-go-lib` to build and compile the go server.

## Usage

To enable the Go online feature retrieval component, set `go_feature_retrieval: True` in your `feature_store.yaml`. This will direct all online feature retrieval to Go instead of Python. This flag will be enabled by default in the future.

{% code title="feature_store.yaml" %}
```yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
type: redis
connection_string: "localhost:6379"
go_feature_retrieval: True
```
{% endcode %}

## Future/Current Work

The Go feature retrieval online feature logging for Data Quality Monitoring is currently in development. More information can be found [here](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit#heading=h.9gaqqtox9jg6).

We also plan on adding support for the Java feature server (e.g. the capability to call into the Go component and execute Java UDFs).

2 changes: 1 addition & 1 deletion sdk/python/feast/feature_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1311,7 +1311,7 @@ def _get_online_features(
}

# If Go feature server is enabled, send request to it instead of going through regular Python logic
if self.config.go_feature_server:
if self.config.go_feature_retrieval:
from feast.embedded_go.online_features_service import (
EmbeddedOnlineFeatureServer,
)
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/feast/repo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ class RepoConfig(FeastBaseModel):

repo_path: Optional[Path] = None

go_feature_server: Optional[bool] = False
go_feature_retrieval: Optional[bool] = False

def __init__(self, **data: Any):
super().__init__(**data)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class IntegrationTestRepoConfig:
full_feature_names: bool = True
infer_features: bool = False
python_feature_server: bool = False
go_feature_server: bool = False
go_feature_retrieval: bool = False

def __repr__(self) -> str:
if not self.online_store_creator:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,13 +116,13 @@
),
# Go implementation for online retrieval
IntegrationTestRepoConfig(
online_store=REDIS_CONFIG, go_feature_server=True,
online_store=REDIS_CONFIG, go_feature_retrieval=True,
),
# TODO(felixwang9817): Enable this test once https://github.com/feast-dev/feast/issues/2544 is resolved.
# IntegrationTestRepoConfig(
# online_store=REDIS_CONFIG,
# python_feature_server=True,
# go_feature_server=True,
# go_feature_retrieval=True,
# ),
]
)
Expand Down Expand Up @@ -154,7 +154,7 @@


GO_REPO_CONFIGS = [
IntegrationTestRepoConfig(online_store=REDIS_CONFIG, go_feature_server=True,),
IntegrationTestRepoConfig(online_store=REDIS_CONFIG, go_feature_retrieval=True,),
]


Expand Down Expand Up @@ -423,7 +423,7 @@ def construct_test_environment(
online_store=online_store,
repo_path=repo_dir_name,
feature_server=feature_server,
go_feature_server=test_repo_config.go_feature_server,
go_feature_retrieval=test_repo_config.go_feature_retrieval,
)

# Create feature_store.yaml out of the config
Expand Down