You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A good point that @jacobwhall brought up is by wrapping prefect tasks we ultimately prevent identifying failed tasks directly via prefect (i.e., the UI) since the final task run always seems to succeed due to the except statement. Not wrapping prefect tasks has implications on the use of prefect futures to generate dependencies across tasks, and would likely require adding code to manage logging for failed prefect tasks.
Using the wrapper for prefect seems like the best short term option, but open to exploring other routes long term.
The text was updated successfully, but these errors were encountered:
Enable retries for prefect tasks via config args to set number of retries and delay between retries.
In addition, we need to implement a separate error wrapper for prefect tasks which allows prefect to manage task failures until the retry cap is reached (and then generates and error wrapper for logging, etc). See: https://github.com/aiddata/geo-datasets/blob/master/malaria_atlas_project/run_tasks.py#L87
A good point that @jacobwhall brought up is by wrapping prefect tasks we ultimately prevent identifying failed tasks directly via prefect (i.e., the UI) since the final task run always seems to succeed due to the except statement. Not wrapping prefect tasks has implications on the use of prefect futures to generate dependencies across tasks, and would likely require adding code to manage logging for failed prefect tasks.
Using the wrapper for prefect seems like the best short term option, but open to exploring other routes long term.
The text was updated successfully, but these errors were encountered: