Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve aws lambda deployment (logging, idempotency, etc) #1985

Merged
merged 2 commits into from
Nov 2, 2021

Conversation

tsotnet
Copy link
Collaborator

@tsotnet tsotnet commented Oct 29, 2021

Signed-off-by: Tsotne Tabidze tsotne@tecton.ai

What this PR does / why we need it: This PR fixes or improves a couple of things

  1. Docker push command was failing for me without any errors in Python, which took quite some time to debug. With this change I make it possible to print out docker command outputs. This is logged on the debug level, so you'll need to run it with --log-level=debug, for example feast --log-level=debug apply. Also, looks like log-level flag was not working correctly, so I also fixed that.
  2. We shouldn't try docker image download/upload if the image is already in ECR. So I made a change that first checks this image in ECR and only runs the docker logic if necessary. This also means that from the 2nd feast apply, users won't need to have docker daemon running (which was required on each apply before).

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Signed-off-by: Tsotne Tabidze <tsotne@tecton.ai>
@codecov-commenter
Copy link

codecov-commenter commented Oct 29, 2021

Codecov Report

Merging #1985 (b110244) into master (6728f2a) will increase coverage by 0.08%.
The diff coverage is 10.14%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1985      +/-   ##
==========================================
+ Coverage   82.07%   82.15%   +0.08%     
==========================================
  Files         100      101       +1     
  Lines        7987     8074      +87     
==========================================
+ Hits         6555     6633      +78     
- Misses       1432     1441       +9     
Flag Coverage Δ
integrationtests 74.62% <10.14%> (+0.16%) ⬆️
unittests 58.94% <10.14%> (-0.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdk/python/feast/cli.py 38.21% <0.00%> (-0.30%) ⬇️
sdk/python/feast/infra/aws.py 39.26% <9.37%> (-0.41%) ⬇️
sdk/python/feast/constants.py 100.00% <100.00%> (ø)
sdk/python/tests/utils/online_read_write_test.py 100.00% <0.00%> (ø)
sdk/python/feast/infra/offline_stores/bigquery.py 80.00% <0.00%> (ø)
...n/feast/infra/feature_servers/aws_lambda/config.py 100.00% <0.00%> (ø)
sdk/python/feast/infra/provider.py 89.43% <0.00%> (+0.08%) ⬆️
.../integration/online_store/test_universal_online.py 99.46% <0.00%> (+0.13%) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6728f2a...b110244. Read the comment docs.

Signed-off-by: Tsotne Tabidze <tsotne@tecton.ai>
Copy link
Collaborator

@felixwang9817 felixwang9817 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great overall, just one quick comment

sdk/python/feast/infra/aws.py Show resolved Hide resolved
Copy link
Collaborator

@adchia adchia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will leave @felixwang9817 to give the lgtm, but the code lgtm

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adchia, felixwang9817, tsotnet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [adchia,felixwang9817,tsotnet]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@felixwang9817
Copy link
Collaborator

/lgtm

@feast-ci-bot feast-ci-bot merged commit bc4ffa5 into feast-dev:master Nov 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants