Skip to content

Commit

Permalink
docs: Add docs for Go feature server (#2540)
Browse files Browse the repository at this point in the history
* Add docs for Go feature server

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

* Update go feature server docs

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Address review components

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Fix

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Fix links

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Revert

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Fix

Signed-off-by: Kevin Zhang <kzhang@tecton.ai>

* Fix test config

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

Co-authored-by: Kevin Zhang <kzhang@tecton.ai>
  • Loading branch information
felixwang9817 and kevjumba authored Apr 14, 2022
1 parent 44f5c11 commit da1ddab
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 7 deletions.
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
* [.feastignore](reference/feature-repository/feast-ignore.md)
* [Feature servers](reference/feature-servers/README.md)
* [Local feature server](reference/feature-servers/local-feature-server.md)
* [Go-based feature retrieval](reference/feature-servers/go-feature-retrieval.md)
* [\[Alpha\] Data quality monitoring](reference/dqm.md)
* [\[Alpha\] On demand feature view](reference/alpha-on-demand-feature-view.md)
* [\[Alpha\] Stream ingestion](reference/alpha-stream-ingestion.md)
Expand Down
38 changes: 38 additions & 0 deletions docs/reference/feature-servers/go-feature-retrieval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Go-based Feature Retrieval

## Overview

The Go Feature Retrieval component is a Go implementation of the core feature serving logic, embedded in the Python SDK. It supports retrieval of feature references, feature services, and on demand feature views, and can be used either through the Python SDK or the [Python feature server](local-feature-server.md).

Currently, this component only supports online serving and does not have an offline component including APIs to create feast feature repositories or apply configuration to the registry to facilitate online materialization. It also does not expose its own dedicated cli to perform feast actions. Furthermore, this component is only meant to expose an online serving API that can be called through the python SDK to facilitate faster online feature retrieval.

The Go Feature Retrieval component currently only supports Redis and Sqlite as online stores; support for other online stores will be added soon. Initial benchmarks indicate that it is significantly faster than the Python feature server for online feature retrieval. We plan to release a more comprehensive set of benchmarks. For more details, see the [RFC](https://docs.google.com/document/d/1Lgqv6eWYFJgQ7LA_jNeTh8NzOPhqI9kGTeyESRpNHnE).

## Installation

As long as you are running macOS or linux x86 with python version 3.7-3.10, the go component comes pre-compiled when you run install feast.

For developers, if you want to build from source, run `make compile-go-lib` to build and compile the go server.

## Usage

To enable the Go online feature retrieval component, set `go_feature_retrieval: True` in your `feature_store.yaml`. This will direct all online feature retrieval to Go instead of Python. This flag will be enabled by default in the future.

{% code title="feature_store.yaml" %}
```yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
type: redis
connection_string: "localhost:6379"
go_feature_retrieval: True
```
{% endcode %}
## Future/Current Work
The Go feature retrieval online feature logging for Data Quality Monitoring is currently in development. More information can be found [here](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit#heading=h.9gaqqtox9jg6).
We also plan on adding support for the Java feature server (e.g. the capability to call into the Go component and execute Java UDFs).
2 changes: 1 addition & 1 deletion sdk/python/feast/feature_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1311,7 +1311,7 @@ def _get_online_features(
}

# If Go feature server is enabled, send request to it instead of going through regular Python logic
if self.config.go_feature_server:
if self.config.go_feature_retrieval:
from feast.embedded_go.online_features_service import (
EmbeddedOnlineFeatureServer,
)
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/feast/repo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ class RepoConfig(FeastBaseModel):

repo_path: Optional[Path] = None

go_feature_server: Optional[bool] = False
go_feature_retrieval: Optional[bool] = False

def __init__(self, **data: Any):
super().__init__(**data)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class IntegrationTestRepoConfig:
full_feature_names: bool = True
infer_features: bool = False
python_feature_server: bool = False
go_feature_server: bool = False
go_feature_retrieval: bool = False

def __repr__(self) -> str:
if not self.online_store_creator:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,13 +116,13 @@
),
# Go implementation for online retrieval
IntegrationTestRepoConfig(
online_store=REDIS_CONFIG, go_feature_server=True,
online_store=REDIS_CONFIG, go_feature_retrieval=True,
),
# TODO(felixwang9817): Enable this test once https://github.com/feast-dev/feast/issues/2544 is resolved.
# IntegrationTestRepoConfig(
# online_store=REDIS_CONFIG,
# python_feature_server=True,
# go_feature_server=True,
# go_feature_retrieval=True,
# ),
]
)
Expand Down Expand Up @@ -154,7 +154,7 @@


GO_REPO_CONFIGS = [
IntegrationTestRepoConfig(online_store=REDIS_CONFIG, go_feature_server=True,),
IntegrationTestRepoConfig(online_store=REDIS_CONFIG, go_feature_retrieval=True,),
]


Expand Down Expand Up @@ -423,7 +423,7 @@ def construct_test_environment(
online_store=online_store,
repo_path=repo_dir_name,
feature_server=feature_server,
go_feature_server=test_repo_config.go_feature_server,
go_feature_retrieval=test_repo_config.go_feature_retrieval,
)

# Create feature_store.yaml out of the config
Expand Down

0 comments on commit da1ddab

Please sign in to comment.