From 58527de5ba848634a997c6517249de19060833af Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Tue, 7 Jan 2020 13:55:04 +0800 Subject: [PATCH] Remove CONTRIBUTING.md (#411) --- CONTRIBUTING.md | 334 ------------------------------------------------ README.md | 11 +- 2 files changed, 6 insertions(+), 339 deletions(-) delete mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index eb38db3008..0000000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,334 +0,0 @@ -# Contributing Guide - -## Getting Started - -The following guide will help you quickly run Feast in your local machine. - -The main components of Feast are: -- **Feast Core** handles FeatureSpec registration, starts and monitors Ingestion - jobs and ensures that Feast internal metadata is consistent. -- **Feast Ingestion** subscribes to streams of FeatureRow and writes the feature - values to registered Stores. -- **Feast Serving** handles requests for features values retrieval from the end users. - -![Feast Components Overview](docs/assets/feast-components-overview.png) - -**Pre-requisites** -- Java SDK version 8 -- Python version 3.6 (or above) and pip -- Access to Postgres database (version 11 and above) -- Access to [Redis](https://redis.io/topics/quickstart) instance (tested on version 5.x) -- Access to [Kafka](https://kafka.apache.org/) brokers (tested on version 2.x) -- [Maven ](https://maven.apache.org/install.html) version 3.6.x -- [grpc_cli](https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md) - is useful for debugging and quick testing -- An overview of Feast specifications and [protos](./protos/feast) - -> **Assumptions:** -> -> 1. Postgres is running in "localhost:5432" and has a database called "postgres" which -> can be accessed with credentials user "postgres" and password "password". -> To use different database name and credentials, please update -> "$FEAST_HOME/core/src/main/resources/application.yml" -> or set these environment variables: DB_HOST, DB_USERNAME, DB_PASSWORD. -> 2. Redis is running locally and accessible from "localhost:6379" -> 3. Feast has admin access to BigQuery. - - -``` -# Clone Feast branch 0.3-dev -# $FEAST_HOME will refer to be the root directory of this Feast Git repository - -git clone -b 0.3-dev https://github.com/gojek/feast -cd feast -``` - -#### Starting Feast Core - -``` -# Please check the default configuration for Feast Core in -# "$FEAST_HOME/core/src/main/resources/application.yml" and update it accordingly. -# -# Start Feast Core GRPC server on localhost:6565 -mvn --projects core spring-boot:run - -# If Feast Core starts successfully, verify the correct Stores are registered -# correctly, for example by using grpc_cli. -grpc_cli call localhost:6565 GetStores '' - -# Should return something similar to the following. -# Note that you should change BigQuery projectId and datasetId accordingly -# in "$FEAST_HOME/core/src/main/resources/application.yml" - -store { - name: "SERVING" - type: REDIS - subscriptions { - project: "*" - name: "*" - version: "*" - } - redis_config { - host: "localhost" - port: 6379 - } -} -store { - name: "WAREHOUSE" - type: BIGQUERY - subscriptions { - project: "*" - name: "*" - version: "*" - } - bigquery_config { - project_id: "my-google-project-id" - dataset_id: "my-bigquery-dataset-id" - } -} -``` - -#### Starting Feast Serving - -Feast Serving requires administrators to provide an **existing** store name in Feast. -An instance of Feast Serving can only retrieve features from a **single** store. -> In order to retrieve features from multiple stores you must start **multiple** -instances of Feast serving. If you start multiple Feast serving on a single host, -make sure that they are listening on different ports. - -``` -# Start Feast Serving GRPC server on localhost:6566 with store name "SERVING" -mvn --projects serving spring-boot:run -Dspring-boot.run.arguments='--feast.store-name=SERVING' - -# To verify Feast Serving starts successfully -grpc_cli call localhost:6566 GetFeastServingType '' - -# Should return something similar to the following. -type: FEAST_SERVING_TYPE_ONLINE -``` - - -#### Registering a FeatureSet - -Create a new FeatureSet on Feast by sending a request to Feast Core. When a -feature set is successfully registered, Feast Core will start an **ingestion** job -that listens for new features in the FeatureSet. Note that Feast currently only -supports source of type "KAFKA", so you must have access to a running Kafka broker -to register a FeatureSet successfully. - -``` -# Example of registering a new driver feature set -# Note the source value, it assumes that you have access to a Kafka broker -# running on localhost:9092 - -grpc_cli call localhost:6565 ApplyFeatureSet ' -feature_set { - name: "driver" - version: 1 - - entities { - name: "driver_id" - value_type: INT64 - } - - features { - name: "city" - value_type: STRING - } - - source { - type: KAFKA - kafka_source_config { - bootstrap_servers: "localhost:9092" - } - } -} -' - -# To check that the FeatureSet has been registered correctly. -# You should also see logs from Feast Core of the ingestion job being started -grpc_cli call localhost:6565 GetFeatureSets '' -``` - - -#### Ingestion and Population of Feature Values - -``` -# Produce FeatureRow messages to Kafka so it will be ingested by Feast -# and written to the registered stores. -# Make sure the value here is the topic assigned to the feature set -# ... producer.send("feast-driver-features" ...) -# -# Install Python SDK to help writing FeatureRow messages to Kafka -cd $FEAST_HOME/sdk/python -pip3 install -e . -pip3 install pendulum - -# Produce FeatureRow messages to Kafka so it will be ingested by Feast -# and written to the corresponding store. -# Make sure the value here is the topic assigned to the feature set -# ... producer.send("feast-test_feature_set-features" ...) -python3 - < Tool Windows > Maven` -1. Drill down to e.g. `Feast Core > Plugins > spring-boot:run`, right-click and `Create 'feast-core [spring-boot'…` -1. In the dialog that pops up, check the `Resolve Workspace artifacts` box -1. Click `OK`. You should now be able to select this run configuration for the Play button in the main toolbar, keyboard shortcuts, etc. - -[idea-boot-main]: https://stackoverflow.com/questions/30237768/run-spring-boots-main-using-ide - -#### Tips for Running Postgres, Redis and Kafka with Docker - -This guide assumes you are running Docker service on a bridge network (which -is usually the case if you're running Linux). Otherwise, you may need to -use different network options than shown below. - -> `--net host` usually only works as expected when you're running Docker -> service in bridge networking mode. - -``` -# Start Postgres -docker run --name postgres --rm -it -d --net host -e POSTGRES_DB=postgres -e POSTGRES_USER=postgres \ --e POSTGRES_PASSWORD=password postgres:12-alpine - -# Start Redis -docker run --name redis --rm -it --net host -d redis:5-alpine - -# Start Zookeeper (needed by Kafka) -docker run --rm \ - --net=host \ - --name=zookeeper \ - --env=ZOOKEEPER_CLIENT_PORT=2181 \ - --detach confluentinc/cp-zookeeper:5.2.1 - -# Start Kafka -docker run --rm \ - --net=host \ - --name=kafka \ - --env=KAFKA_ZOOKEEPER_CONNECT=localhost:2181 \ - --env=KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \ - --env=KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ - --detach confluentinc/cp-kafka:5.2.1 -``` - -## Code reviews - -Code submission to Feast (including submission from project maintainers) requires review and approval. -Please submit a **pull request** to initiate the code review process. We use [prow](https://github.com/kubernetes/test-infra/tree/master/prow) to manage the testing and reviewing of pull requests. Please refer to [config.yaml](../.prow/config.yaml) for details on the test jobs. - -## Code conventions - -### Java - -We conform to the [Google Java Style Guide]. Maven can helpfully take care of -that for you before you commit: - - $ mvn spotless:apply - -Formatting will be checked automatically during the `verify` phase. This can be -skipped temporarily: - - $ mvn spotless:check # Check is automatic upon `mvn verify` - $ mvn verify -Dspotless.check.skip - -If you're using IntelliJ, you can import [these code style settings][G -IntelliJ] if you'd like to use the IDE's reformat function as you work. - -### Go - -Make sure you apply `go fmt`. - -[Google Java Style Guide]: https://google.github.io/styleguide/javaguide.html -[G IntelliJ]: https://github.com/google/styleguide/blob/gh-pages/intellij-java-google-style.xml diff --git a/README.md b/README.md index d9b1674826..ef49427497 100644 --- a/README.md +++ b/README.md @@ -29,11 +29,12 @@ prediction = my_model.predict(fs.get_online_features(customer_features, customer ``` ## Important resources - * [Why Feast?](docs/why-feast.md) - * [Concepts](docs/concepts.md) - * [Installation](docs/getting-started/installing-feast.md) - * [Getting Help](docs/community.md) + * [Why Feast?](https://docs.feast.dev/why-feast) + * [Concepts](https://docs.feast.dev/concepts) + * [Installation](https://docs.feast.dev/getting-started/installing-feast) + * [Getting Help](https://docs.feast.dev/getting-help) + * [Example Notebook](https://github.com/gojek/feast/blob/master/examples/basic/basic.ipynb) ## Notice -Feast is a community project and is still under active development. Your feedback and contributions are important to us. Please have a look at our [contributing guide](CONTRIBUTING.md) for details. +Feast is a community project and is still under active development. Your feedback and contributions are important to us. Please have a look at our [contributing guide](docs/contributing.md) for details.