Github workflow badge showing failure when is not the case

[mondeja]

Are you experiencing an issue with...

shields.io

🐞 Description

Hi,

After updating Github workflow from the old URL:

https://img.shields.io/github/workflow/status/simple-icons/simple-icons/Verify/develop?logo=github&label=tests

to the new API

https://img.shields.io/github/actions/workflow/status/simple-icons/simple-icons/verify.yml?branch=develop&logo=github&label=tests

The badge is showing that the tests are failing, which is not the case as the latest workflow for develop branch is passing the CI.

🔗 Link to the badge

https://img.shields.io/github/actions/workflow/status/simple-icons/simple-icons/verify.yml?branch=develop&logo=github&label=tests

💡 Possible Solution

No response

[chris48s]

Thanks for reporting. My suggestion for now is to use https://img.shields.io/github/actions/workflow/status/simple-icons/simple-icons/verify.yml?branch=develop&logo=github&label=tests&event=push but I need to dig into this one more.

[chris48s]

OK. The issue here is that branch=develop is also bringing back builds from pull requests where the branch is called develop, so we're reporting the first build status shown here, rather than the third one (which is what you'd expect).

I'll have a look at a fix for this one tomorrow

[chris48s]

Of the various issues that have come out of this, this is the one that is worrying me the most as we are serving the wrong data in some situations and I can't see a completely obvious fix.

As far as I can tell, there is no way to filter to only branches called develop that are in the target repo. I was hoping something like

• https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=simple-icons%2Fsimple-icons%3Adevelop&page=1&exclude_pull_requests=true or
• https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=refs%2Fheads%3Adevelop&page=1&exclude_pull_requests=true
  might do the job, but no dice. This is unexpected, but as far as I can see there is not much we can do about it. I think I am going to raise an issue with GitHub support, but I suspect the answer is going to be "you need to pass event=push"

I think there are a few options, all of which are imperfect in one way or another:

1. Instead of getting the first result (with per_page=1)..

   • if event is not specified, we request the first N results and then pick the first one where head_repository matches the :user/:repo from the route
   • if event is specified, just pick the first one

   I think the big disadvantage of this approach is performance. Compare the speed of fetching https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=develop&page=1&per_page=1&exclude_pull_requests=true to https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=develop&page=1&per_page=300&exclude_pull_requests=true for example. I worry this could push us over the crucial 4 seconds in a lot of cases.

2. If event is not specified, silently default to push
   This is probably a reasonable-ish assumption for most cases where the user wants a status badge, but will break badges for some subset of users who are happily using it to report the status of a workflow that runs on worflow_dispatch, schedule, deployment or whatever without having to specify ?event=. I don't have too much of an appetite to make another breaking change to this badge at this point.

3. Switch the badge from using the API to being a 'scraper' for GitHub's native badge (the version which uses the workflow filename) that just behaves exactly the same way as the native one but allows the shields style customisations (styles, colours, icons, ets).
   I've tested the native badge, and it doesn't have this problem. Having had a bit of a experiment over in https://github.com/chris48s/workflow-badge-repro , I think it is basically impossible for us to precisely recreate the behaviour of this using a single API call to the workflow /runs endpoint. Doing this option would make everyone happy who is affected by Github Actions Workflow badge not working for "branchless" push #8736 too. The one thing we lose by doing that (instead of using the API) is that the people who wanted the ability to use this badge for private repos on a self-hosted shields instance won't be able to have it, but we've yet to put out a tagged docker release with a changelog entry saying that feature exists, so I feel like we're not really pulling the rug out from anyone yet if we do that.

4. There is maybe another slightly more nuanced approach where we fetch the workflow file, parse it as yaml, work out what event triggers it uses and then do something sensible based on that. That would be 2 requests but might possibly still faster than option 1?

Although they are two different issues, I think I'd like to also consider #8736 in working out what we do next. My instinct at this point is to go with option 3. All ears if anyone else has a better plan.

[chris48s]

I've submitted a support ticket to GitHub asking

If I make a call to
https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=develop&exclude_pull_requests=true
this returns both

• Runs of verify.yml on pushes to the develop branch of https://github.com/simple-icons/simple-icons/
• Runs of verify.yml on pull requests that were submitted to https://github.com/simple-icons/simple-icons/ where the user submitting the pull request made it from the develop branch of their fork of the https://github.com/simple-icons/simple-icons/ repo

I understand what is happening here, and it is the same behaviour we see on https://github.com/simple-icons/simple-icons/actions/workflows/verify.yml?query=branch%3Adevelop but it is also somewhat unexpected.

Is it possible to filter this such that the branch=develop filter only applies to branches of the https://github.com/simple-icons/simple-icons/ repo (and not forks) without also filtering on event=push?

I was hoping something like
https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=simple-icons%2Fsimple-icons%3Adevelop&page=1&exclude_pull_requests=true
or
https://api.github.com/repos/simple-icons/simple-icons/actions/workflows/verify.yml/runs?branch=refs%2Fheads%3Adevelop&page=1&exclude_pull_requests=true
might work, but no joy.

As I say, I suspect the answer will be no but lets see..

[chris48s]

Response from GH support confirming this is not possible:

Thank you for contacting GitHub Support.

This is the expected behavior. Workflow run filter by branches returns all branches that contain the keyword.

We do not support filtering to only exact matches.

We appreciate your feedback! We're always working to improve GitHub and we consider every suggestion we receive.

We also invite you to share this feedback as a feature request in our official GitHub public feedback discussions repository for better visibility.

https://github.com/community/community/discussions

I hope this helps, please let us know if you need any further assistance.

I may follow up on submitting a feature request, but this isn't moving any time soon. I think I'm pretty convinced that #8738 is the best option.

[chris48s]

We've just merged #8738 and deployed to production, which fixes this bug.