Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Bytewax materializer security context #3573

Merged
merged 3 commits into from
Apr 21, 2023
Merged

fix: Bytewax materializer security context #3573

merged 3 commits into from
Apr 21, 2023

Conversation

adamschmidt
Copy link
Contributor

What this PR does / why we need it:

This PR adds a configuration option that signals whether or not to include Kubernetes security context capabilities in the job & pod specification.

Metadata labels also added to assist with cleaning up resources after materialization jobs have completed.

Which issue(s) this PR fixes:

Fixes #3572

@adamschmidt
fix: Bytewax materializer security context
c7617d3 
Signed-off-by: adamschmidt <aschmidt1978@gmail.com>
@adamschmidt
fix: lint
5931aef 
Signed-off-by: adamschmidt <aschmidt1978@gmail.com>
@adamschmidt
chore: docs
bb84fc8 
Signed-off-by: adamschmidt <aschmidt1978@gmail.com>
@adamschmidt
Copy link
Contributor Author

Hey @whoahbot - one quick update to satisfy tighter Kubernetes runtime requirements. I've tested this in my local Kubernetes environment, runs without issues.

@whoahbot
Copy link
Collaborator

Thanks @adamschmidt

/lgtm

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adamschmidt, whoahbot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@adamschmidt
Copy link
Contributor Author

Thanks for the quick review @whoahbot!

@adamschmidt
Copy link
Contributor Author

@whoahbot @achals The integration tests seem to be intermittently failing for unrelated reasons. Is there something wrong with the integration test setup? I've not experienced this.

@adamschmidt
Copy link
Contributor Author

@whoahbot @achals - Bump on this. Are we able to re-run the tests and close this? I'm unsure why a BigQuery test is failing when this change does not touch that?

@adchia
Copy link
Collaborator

adchia commented Apr 21, 2023

going to just merge this since it looks to be a flaky test

@adchia adchia merged commit 6794338 into feast-dev:master Apr 21, 2023
felixwang9817 pushed a commit that referenced this pull request Apr 21, 2023
@feast-ci-bot
chore(release): release 0.31.0
902f23f 
# [0.31.0](v0.30.0...v0.31.0) (2023-04-21)

### Bug Fixes

* Add Stream Feature Views to helper that collect Feature View names ([#3582](#3582)) ([7854f63](7854f63))
* Add StreamFeatureViewSpec to FeastObjectSpecProto convenience type ([#3550](#3550)) ([3cefd6c](3cefd6c))
* Batch Snowflake materialization queries to obey Snowpark 100 fea… ([#3406](#3406)) ([f9862b5](f9862b5))
* Bytewax materializer security context ([#3573](#3573)) ([6794338](6794338))
* **cI:** Install coreutils in mac github workers for smoke test ([#3563](#3563)) ([e7421c1](e7421c1))
* Fix bug with no SqlRegistryConfig class ([#3586](#3586)) ([6dc1368](6dc1368))
* Fix Snowflake template ([#3584](#3584)) ([6c09c39](6c09c39))
* Make snowflake to remote tables temporary ([#3588](#3588)) ([ad48146](ad48146))
* Remove snowflake source warehouse tech debt ([#3422](#3422)) ([7da0580](7da0580))
* Snowflake remote storage ([#3574](#3574)) ([f8d3890](f8d3890))
* Support param timeout when persisting ([#3593](#3593)) ([01a98f0](01a98f0))
* Use pyarrow in a way that works across versions ([#3562](#3562)) ([1289f3f](1289f3f))
* Wrap the bigquery table name with backtick. ([#3577](#3577)) ([09f0e7e](09f0e7e))

### Features

* Add AWS Redshift Serverless support ([#3595](#3595)) ([58ce148](58ce148))
* Add Hazelcast as an online store ([#3523](#3523)) ([b05d50b](b05d50b))
* Cache Bigtable client ([#3602](#3602)) ([b27472f](b27472f))
* Relax aws extras requirements ([#3585](#3585)) ([7e77382](7e77382))
* Show bigquery datasource table and query on UI ([#3600](#3600)) ([58d63f7](58d63f7))
* Update snowflake offline store job output formats -- added arrow ([#3589](#3589)) ([be3e349](be3e349))
zerafachris pushed a commit to zerafachris/feast that referenced this pull request Mar 5, 2024
@adamschmidt @zerafachris
fix: Bytewax materializer security context (feast-dev#3573)
ed8624e 
* fix: Bytewax materializer security context

Signed-off-by: adamschmidt <aschmidt1978@gmail.com>

* fix: lint

Signed-off-by: adamschmidt <aschmidt1978@gmail.com>

* chore: docs

Signed-off-by: adamschmidt <aschmidt1978@gmail.com>

---------

Signed-off-by: adamschmidt <aschmidt1978@gmail.com>
Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
zerafachris pushed a commit to zerafachris/feast that referenced this pull request Mar 5, 2024
@feast-ci-bot @zerafachris
chore(release): release 0.31.0
e7cc13a 
# [0.31.0](feast-dev/feast@v0.30.0...v0.31.0) (2023-04-21)

### Bug Fixes

* Add Stream Feature Views to helper that collect Feature View names ([feast-dev#3582](feast-dev#3582)) ([7854f63](feast-dev@7854f63))
* Add StreamFeatureViewSpec to FeastObjectSpecProto convenience type ([feast-dev#3550](feast-dev#3550)) ([3cefd6c](feast-dev@3cefd6c))
* Batch Snowflake materialization queries to obey Snowpark 100 fea… ([feast-dev#3406](feast-dev#3406)) ([f9862b5](feast-dev@f9862b5))
* Bytewax materializer security context ([feast-dev#3573](feast-dev#3573)) ([6794338](feast-dev@6794338))
* **cI:** Install coreutils in mac github workers for smoke test ([feast-dev#3563](feast-dev#3563)) ([e7421c1](feast-dev@e7421c1))
* Fix bug with no SqlRegistryConfig class ([feast-dev#3586](feast-dev#3586)) ([6dc1368](feast-dev@6dc1368))
* Fix Snowflake template ([feast-dev#3584](feast-dev#3584)) ([6c09c39](feast-dev@6c09c39))
* Make snowflake to remote tables temporary ([feast-dev#3588](feast-dev#3588)) ([ad48146](feast-dev@ad48146))
* Remove snowflake source warehouse tech debt ([feast-dev#3422](feast-dev#3422)) ([7da0580](feast-dev@7da0580))
* Snowflake remote storage ([feast-dev#3574](feast-dev#3574)) ([f8d3890](feast-dev@f8d3890))
* Support param timeout when persisting ([feast-dev#3593](feast-dev#3593)) ([01a98f0](feast-dev@01a98f0))
* Use pyarrow in a way that works across versions ([feast-dev#3562](feast-dev#3562)) ([1289f3f](feast-dev@1289f3f))
* Wrap the bigquery table name with backtick. ([feast-dev#3577](feast-dev#3577)) ([09f0e7e](feast-dev@09f0e7e))

### Features

* Add AWS Redshift Serverless support ([feast-dev#3595](feast-dev#3595)) ([58ce148](feast-dev@58ce148))
* Add Hazelcast as an online store ([feast-dev#3523](feast-dev#3523)) ([b05d50b](feast-dev@b05d50b))
* Cache Bigtable client ([feast-dev#3602](feast-dev#3602)) ([b27472f](feast-dev@b27472f))
* Relax aws extras requirements ([feast-dev#3585](feast-dev#3585)) ([7e77382](feast-dev@7e77382))
* Show bigquery datasource table and query on UI ([feast-dev#3600](feast-dev#3600)) ([58d63f7](feast-dev@58d63f7))
* Update snowflake offline store job output formats -- added arrow ([feast-dev#3589](feast-dev#3589)) ([be3e349](feast-dev@be3e349))

Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@whoahbot whoahbot whoahbot approved these changes

Assignees

@whoahbot whoahbot

Labels
approved lgtm ok-to-test size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Bytewax Materializer Security Context Capabilities
5 participants
@adamschmidt @whoahbot @feast-ci-bot @adchia @achals