Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Updated feast Go operator db stores #4809

Merged
merged 2 commits into from
Dec 4, 2024
Merged

feat: Updated feast Go operator db stores #4809

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1b39951
feat: Updated feast Go operator db stores
tmihalac Dec 4, 2024
a5e0b8e
Updated sample secret following review
tmihalac Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions infra/feast-operator/api/v1alpha1/featurestore_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,11 @@ var ValidOfflineStoreFilePersistenceTypes = []string{
// OfflineStoreDBStorePersistence configures the DB store persistence for the offline store service
type OfflineStoreDBStorePersistence struct {
// +kubebuilder:validation:Enum=snowflake.offline;bigquery;redshift;spark;postgres;feast_trino.trino.TrinoOfflineStore;redis
Type string `json:"type"`
SecretRef corev1.LocalObjectReference `json:"secretRef"`
SecretKeyName string `json:"secretKeyName,omitempty"`
Type string `json:"type"`
// Data store parameters should be placed as-is from the "feature_store.yaml" under the secret key. "registry_type" & "type" fields should be removed.
SecretRef corev1.LocalObjectReference `json:"secretRef"`
// By default, the selected store "type" is used as the SecretKeyName
SecretKeyName string `json:"secretKeyName,omitempty"`
}

var ValidOfflineStoreDBStorePersistenceTypes = []string{
Expand Down Expand Up @@ -149,9 +151,11 @@ type OnlineStoreFilePersistence struct {
// OnlineStoreDBStorePersistence configures the DB store persistence for the offline store service
type OnlineStoreDBStorePersistence struct {
// +kubebuilder:validation:Enum=snowflake.online;redis;ikv;datastore;dynamodb;bigtable;postgres;cassandra;mysql;hazelcast;singlestore
Type string `json:"type"`
SecretRef corev1.LocalObjectReference `json:"secretRef"`
SecretKeyName string `json:"secretKeyName,omitempty"`
Type string `json:"type"`
// Data store parameters should be placed as-is from the "feature_store.yaml" under the secret key. "registry_type" & "type" fields should be removed.
SecretRef corev1.LocalObjectReference `json:"secretRef"`
// By default, the selected store "type" is used as the SecretKeyName
SecretKeyName string `json:"secretKeyName,omitempty"`
}

var ValidOnlineStoreDBStorePersistenceTypes = []string{
Expand Down Expand Up @@ -196,9 +200,11 @@ type RegistryFilePersistence struct {
// RegistryDBStorePersistence configures the DB store persistence for the registry service
type RegistryDBStorePersistence struct {
// +kubebuilder:validation:Enum=sql;snowflake.registry
Type string `json:"type"`
SecretRef corev1.LocalObjectReference `json:"secretRef"`
SecretKeyName string `json:"secretKeyName,omitempty"`
Type string `json:"type"`
// Data store parameters should be placed as-is from the "feature_store.yaml" under the secret key. "registry_type" & "type" fields should be removed.
SecretRef corev1.LocalObjectReference `json:"secretRef"`
// By default, the selected store "type" is used as the SecretKeyName
SecretKeyName string `json:"secretKeyName,omitempty"`
}

var ValidRegistryDBStorePersistenceTypes = []string{
Expand Down
52 changes: 34 additions & 18 deletions infra/feast-operator/config/crd/bases/feast.dev_featurestores.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,11 +321,13 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the secret
key. "registry_type" & "type" fields should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -676,11 +678,13 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the secret
key. "registry_type" & "type" fields should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1049,11 +1053,14 @@ spec:
the DB store persistence for the registry service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1517,11 +1524,14 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1878,11 +1888,14 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -2259,11 +2272,14 @@ spec:
the DB store persistence for the registry service
properties:
secretKeyName:
description: By default, the selected store
"type" is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should
be placed as-is from the "feature_store.yaml"
under the secret key. "registry_type" &
"type" fields should be removed.
properties:
name:
description: |-
Expand Down
31 changes: 29 additions & 2 deletions infra/feast-operator/config/samples/v1alpha1_featurestore_db_persistence.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,32 @@ spec:
store:
type: postgres
secretRef:
name: my-secret
secretKeyName: mykey # optional
name: postgres-secret
secretKeyName: postgres-secret-parameters # optional
registry:
local:
persistence:
store:
type: sql
secretRef:
name: postgres-secret
secretKeyName: postgres-secret-parameters # optional
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
secretKeyName: postgres-secret-parameters # optional
secretKeyName: postgres-secret-parameters # optional, will use store.type by default... in this case that's "sql"

---
apiVersion: v1
kind: Secret
metadata:
name: postgres-secret
stringData:
postgres-secret-parameters: |
path: postgresql+postgresql://postgres:mysecretpassword@127.0.0.1:55001/feast
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
path: postgresql+postgresql://postgres:mysecretpassword@127.0.0.1:55001/feast
path: postgresql+psycopg://postgres:mysecretpassword@127.0.0.1:55001/feast

cache_ttl_seconds: 60
sqlalchemy_config_kwargs:
echo: false
pool_pre_ping: true
postgres: |
host: 127.0.0.1
port: 55001
database: feast
db_schema: public
user: postgres
password: mysecretpassword
52 changes: 34 additions & 18 deletions infra/feast-operator/dist/install.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,11 +329,13 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the secret
key. "registry_type" & "type" fields should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -684,11 +686,13 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the secret
key. "registry_type" & "type" fields should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1057,11 +1061,14 @@ spec:
the DB store persistence for the registry service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1525,11 +1532,14 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -1886,11 +1896,14 @@ spec:
the DB store persistence for the offline store service
properties:
secretKeyName:
description: By default, the selected store "type"
is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should be placed
as-is from the "feature_store.yaml" under the
secret key. "registry_type" & "type" fields
should be removed.
properties:
name:
description: |-
Expand Down Expand Up @@ -2267,11 +2280,14 @@ spec:
the DB store persistence for the registry service
properties:
secretKeyName:
description: By default, the selected store
"type" is used as the SecretKeyName
type: string
secretRef:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
description: Data store parameters should
be placed as-is from the "feature_store.yaml"
under the secret key. "registry_type" &
"type" fields should be removed.
properties:
name:
description: |-
Expand Down
Loading
Loading