Frequent test failures caused by image download timeout

[Garett-MacGowan]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issues exists when I tested with :latest
• I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?

This is a development specific issue. When running the CI, tests frequently (intermittently) fail due to a timeout of the argoexec image download.

Here's an example failed workflow

This issue seems to be common & is being experienced by multiple users of the v4 action: actions/download-artifact#249

Potential Solutions

1. Wrap actions/download-artifact@v4 in a retry action.
2. Downgrade to actions/download-artifact@v3
3. Wait for a bug fix for actions/download-artifact@v4 ([bug] Unable to download artifact(s): Unable to download and extract artifact: Request timeout actions/download-artifact#249)

Version

latest

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

N/A

Logs from the workflow controller

N/A

Logs from in your workflow's wait container

N/A

[Garett-MacGowan]

If someone provides some feedback here on the preferred solution, I can quickly implement & push.

[Garett-MacGowan]

@juliev0 FYI I am observing a failure rate much worse than ~1/3 at the moment.

[agilgur5]

Nice catch!

I would suggest we revert #12384 as it seems like most folks in that issue are doing. We'll want to add an ignore pattern to dependabot.yml for that too so that it doesn't get auto-updated.

Re: retry, we may want to add that anyway as it does still occasionally happen even on v3 (though ideally the action itself would have code to retry...), though it seems to be happening exponentially more with actions/download-artifact@v4. So revert would be top priority to fix this and retry would be a nice to have but not necessary, IMO

[Garett-MacGowan]

Looks like the download action already implements a retry, so maybe something else is messed up with the new v4.

I think I'll just revert changes from #12384 and open a new ticket to upgrade to v4 once actions/download-artifact#249 is resolved.

[agilgur5]

Sounds good. We will still want to add the dependabot.yml ignore temporarily until v4 is fixed

[Garett-MacGowan]

Sounds good. We will still want to add the dependabot.yml ignore temporarily until v4 is fixed

Yep, I've got that in. Will push shortly.