Replies: 23 comments 28 replies
-
This perhaps depends on perspective. I had problems to understand At the end of the day I understand it that Why the check is red with the PR then I never fully understood, and I only explain it to myself it is using a steps context From my perspective that difference (Actions vs. Checks) is the main bummer for And if it helps, for seeing if |
Beta Was this translation helpful? Give feedback.
-
This is indeed a common technique I used frequently on Travis CI and Appveyor. Here are links directly to the documentation stating one clear use case that this is for: https://www.appveyor.com/docs/build-configuration/#allow-failing-jobs
This is a useful technique for saying "test against the latest language level" which can change over time and can potentially fail, without violating any documented requirements of a project. In this case one does not want the sum of their checks to be "failed" or "failure", but "ok" or "passing". The yellow indicator requests related to this ask I find are not really necessary, but would not be harmful, and could be added at a later time. https://docs.travis-ci.com/user/customizing-the-build/#jobs-that-are-allowed-to-fail
Additionally I have another use case, which is to add tests for known failures and run the tests in the future to see if they get fixed:
The semantics of
All other dependencies such as step and job dependencies should continue to work as normal. |
Beta Was this translation helpful? Give feedback.
-
Not having a proper |
Beta Was this translation helpful? Give feedback.
-
"Allow failures" as an enabler of incremental improvement(reposting here from 399 as requested by @pboling) I have a one-person project [1] , so PR checks are not an issue for me. However, as with most in this ticket, I am need of "allow failures". Let me add my use case to the discussion as I think it is slightly different from the ones described so far, but probably a very common one. At present in my project we do not support In fact, by not having this feature, it makes incremental improvements to the build much harder. I now need to enable the build, deal with any rotting that may have happened since the last time it was enabled, make some changes, run the build and then disable it again. The drive-by approach I used to take - whenever I got 20 minutes here and there, do a quick experiment to see if it improved things - has now been rendered unworkable because most of those twenty minutes would be spent on overhead. All of this to say that, for me, this is an important feature. Links[1] https://github.com/MASD-Project/dogen |
Beta Was this translation helpful? Give feedback.
-
I have the same problem... My slack action is failing, failing the whole pipeline, although the rest worked... So the PR is locked... At this point, it's hard to believe Github Management is way too far from the community... This feature is definitely a very important ask and would allow us to bypass situations like the one described above... |
Beta Was this translation helpful? Give feedback.
-
Plus one for all of the above. I'm working on fixing flaky tests in our code that cause CI to fail, which means I'm running many, many tests and constantly checking if they succeeded or failed. We have a GitHub check for CI and another GitHub check to flag tests that failed once but succeeded on retry. The "succeeded on retry" check is optional. My work would be greatly facilitated by the optional check not causing the build to be marked a fail. |
Beta Was this translation helpful? Give feedback.
-
Echoing what others are saying here, our use-case is very similar and connected to https://github.com/actions/toolkit/issues/399#issuecomment-1260017446.
|
Beta Was this translation helpful? Give feedback.
-
I agree this would be a nice to have. +1 the |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
Update: The closed issue has been moved to the appropriate, relevant, repository, and is where backlinks should point, even though it remains closed. Now that this is the official, and only, forum for discussion of this issue it is worth point out that - this is a terrible forum for discussing this issue. Github Discussions do not backlink to commits / PRs / Issues that link to them. GitHub issues, OTOH, do offer backlinks. Many, nay hundreds, nay the majority of interactions on the now-closed-issue were backlinks where people were referencing the issue from all over GitHub precisely in the context where they were having problems that would be solved by this feature. Now any future context of that sort will be lost, and we will only get a fraction of the context, discussion, and input from the community, and devoid of the hundreds of real world examples that were available before. Perhaps it is good to have the loudest voice control the discussion (GitHub in this case, speaking over everyone else by saying nothing at all). But if that is the case, don't elfing call it "Community Discussions". It's misleading, as it implies to most reasonable people that GitHub would be participating in the discussion. Call it "The Place To Vent While GitHub Ignores You" instead. |
Beta Was this translation helpful? Give feedback.
-
Regardless of whether ...of course a ton of people have said this already, I just wanted to add my name to do what I can to underscore the importance of this... |
Beta Was this translation helpful? Give feedback.
-
Hey folks 👋 I wrote a post about this on the now infamous closed issue on this topic a while ago where I was sharing a project that I created to bring back some semblance of the "allow failures" feature that we used heavily in the Ember community when Travis was the default CI provider. I thought I would share a link to the project here in case it helps anyone 👍 |
Beta Was this translation helpful? Give feedback.
-
Seeing a red X against a PR that has successfully passed CI for all shipping versions of the underlying runtime, but failed the nightly is requiring additional training of all project maintainers. I'm very surprised that GitHub closed issue actions/runner#2347 as "Not planned" with no notice of why this UX problem is considered acceptable. |
Beta Was this translation helpful? Give feedback.
-
Hi all 👋 - I'm a PM for Actions and looking into this issue. I'm hoping to setup 5-10 conversations with folks to dig into the problem and use cases. If you're interested in chatting, please use Edit on Mar 3, 2023: Thanks for signing up and the responses! I have enough conversations lined up so I've closed the form. I'll reach back out to setup more conversations if needed. |
Beta Was this translation helpful? Give feedback.
-
All I ever wanted is a new yellow icon to show when a job that is allowed to fail failed but does not necessarily block merging. ❌ is too much and ✔️ is misleading. |
Beta Was this translation helpful? Give feedback.
-
As a temporary workaround, if that can help anyone:
I use this in a E2E test logic where we want 3 retries before calling a test "fully failed" such as:
|
Beta Was this translation helpful? Give feedback.
-
This is much needed. There is no way to deem workflow as "skipped" or "neutral" when one of the jobs fails but it is not so important. |
Beta Was this translation helpful? Give feedback.
-
+1 for this feature |
Beta Was this translation helpful? Give feedback.
-
how hard can it be to implement this? is it being blocked on purpose or what? |
Beta Was this translation helpful? Give feedback.
-
+1 for this feature |
Beta Was this translation helpful? Give feedback.
-
How is this feature not supported yet? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
I got sufficiently tired of this that I came up with a work-around. Posting it here in case others find it useful:
What I did gets me:
|
Beta Was this translation helpful? Give feedback.
-
continue-on-error
is a very useful part of GitHub Actions, but unfortunately it's reflected extremely poorly (= not at all) in the UI and specifically in the pull request UI.That means it's near-useless or barely better than
|| true
, which is not the desired feature.The desired feature is to allow some jobs to fail, and indicate it on the PR UI in a sensible way, much like TravisCI did with
allow-failures: true
.continue-on-error
:Check failure on line 1 in .github
but that doesn't link to the proper place.That's not enough notification, and there seem to be no reasonable way to see if that job actually passed or failed.
Example: https://github.com/eregon/sequel/runs/1476292451?check_suite_focus=true#step:7:27
I believe it would be far more useful if
continue-on-error
at the step level would behave more like it currently does at the job level, and mark the CI job as failed or failed-but-expected.The failed step itself should have a red or yellow mark, so it can be noticed without having to guess which step failed.
GitHub Actions already knows about the outcome of every step, and that's also in API, the only missing part is showing it in a useful manner (and not exactly the same as a successful step).
Actions
tab. But on the PR it's rather confusing, and it looks like the CI broke when it did not.Example: https://github.com/jeremyevans/sequel/pull/1737/checks?check_run_id=1479678900
The big red
Some checks were not successful
sounds like something broke, but no, an error/failure there is expected so it shouldn't be marked as "CI failed". And on the wheel there is a bit of red.The whole area is encircled in yellow (I'm not sure why), which seems OK.
So there I would propose to mark the failed job in yellow, and possibly use another symbol. And then to use the same color in the wheel for jobs which are
continue-on-error: true
. And also change the text so it would be something likeAll expected checks have passed
in green (the wording for all checks are green isAll checks have passed
).The whole area should then be encircled in green, because it's safe to merge without looking at logs.
This from issue https://github.com/actions/toolkit/issues/399, which seems to not be getting too little attention from github while there are 100+ comments and 500+ interactions.
Somewhat related discussion: #9875
Beta Was this translation helpful? Give feedback.
All reactions