Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to rerun the failed node in pipeline #536

Open
zhangjinrui2718 opened this issue Jan 5, 2023 · 3 comments
Open

how to rerun the failed node in pipeline #536

zhangjinrui2718 opened this issue Jan 5, 2023 · 3 comments

Comments

@zhangjinrui2718
Copy link

  1. how to retry the failed node in a completed-failed pipeline?
  2. could i add some parameter like 'run date' or another into the pipeline then used in the pipeline? ( such parameter never changed once the a pipeline instance start , also in retry mode)
@saul-data
Copy link
Contributor

For question 1, we need to add in that feature to re run a node or re run the rest of the pipeline if it fails at a certain point. It is on our to do list.

Question 2: There is an environment variable called DP_RUNID that is unique UUID used across the run. You can call that using:

import os
runid = os.getenv("DP_RUNID")

@zhangjinrui2718
Copy link
Author

glad to see ur response , i m expecting the new feature , when will it be published

now pipeline has no stateful information , so that i cant re run the certain instance.
in my thought, the pipeline is just a logic DAG, running it will create a instance with stateful information , 'run date' or 'biz date' is an example and is very important/useful concept related to the instance

@marconimedeiros2
Copy link

After all, is there a rerun feature or not?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants