Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove feature set status check for job update requirement #708

Merged
merged 1 commit into from
May 15, 2020

Conversation

khorshuheng
Copy link
Collaborator

What this PR does / why we need it:

Currently, a job is deemed to require update if one or more associated feature set is in pending state. This leads to infinite loop in which the job manager keep updating existing task but the job status never becomes successful:

  1. New feature set is created, and status is pending.
  2. Job submitted.
  3. In the next job polling, requireUpdate is called, and return true because status of the feature set is pending.
  4. Dataflow job is updated, eventhough it's not supposed to.
  5. Job status update is skipped.

Feature set equality check should have sufficed to determine if a job requires update.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:


@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow

@khorshuheng
Copy link
Collaborator Author

/test test-end-to-end

@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow

@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow

@khorshuheng khorshuheng force-pushed the fix-job-update-loop branch 2 times, most recently from 96534ef to 528a16c Compare May 15, 2020 05:16
@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow

@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow

@zhilingc
Copy link
Collaborator

/test test-end-to-end-batch-dataflow
/test test-end-to-end-batch

@feast-ci-bot
Copy link
Collaborator

feast-ci-bot commented May 15, 2020

@khorshuheng: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
test-end-to-end-batch-dataflow ae35fe9 link /test test-end-to-end-batch-dataflow

Full PR test history

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@zhilingc
Copy link
Collaborator

/lgtm

@zhilingc
Copy link
Collaborator

/approve

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: khorshuheng, zhilingc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [khorshuheng,zhilingc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@zhilingc zhilingc merged commit abb53e2 into feast-dev:master May 15, 2020
@ches ches added the backport-candidate Changes that may be desired for backport to earlier Feast release tracks label May 15, 2020
@ches ches removed the backport-candidate Changes that may be desired for backport to earlier Feast release tracks label May 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants