[test] Disable codecov upload on PR builds #2541
Conversation
|
The expected behavior is that Codecov reports CI failure (for the linked report). |
Codecov Report
@@ Coverage Diff @@
## master #2541 +/- ##
===========================================
+ Coverage 47.38% 78.35% +30.97%
===========================================
Files 84 84
Lines 5960 5960
===========================================
+ Hits 2824 4670 +1846
+ Misses 3136 1290 -1846
Continue to review full report at Codecov.
|
|
I'm not entirely convinced that this fix works, I'm going to leave this PR open to do some more testing over the next week or so. |
Codecov has been returning weird results for some builds, usually trending lower (saying that code coverage has dropped by some ridiculous amount). I think this is because Codecov is getting confused by all the different reports being uploaded - five from the PR build, and another five from the branch build. Here's an example of Codecov conflating reports from different builds: https://codecov.io/gh/HumanCellAtlas/data-store/commit/2fcdaa2c27de001e6a436371950d0caecdb40d3d/builds
natanlao
force-pushed
the
natanlao-fix-codecov
branch
from
406e24b
to
dd8cb36
Compare
| @@ -0,0 +1,3 @@ | |||
| codecov: | |||
| notify: | |||
| after_n_builds: 5 | |||
There was a problem hiding this comment.
Where does the 5 come from? The codecov.io documentation of after_n_builds says it should be set greater than the number of reports uploaded per commit
There was a problem hiding this comment.
I read
Generally, you should set
after_n_buildsequal to the number of reports you upload per commit.
There are five codecov reports (one per job) uploaded per build, so I think this is right. One part I am unclear of is if it refers to number of reports uploaded per commit or per build - if it is the former, then, were we not to disable codecov on one of two builds per PR, after_n_builds should be set to ten. But I think with these changes it might not matter...
There was a problem hiding this comment.
Will keep an eye out for whether the reported statistics improve - if not we can try bumping from 5 to 10.
|
Hmm, the old codecov reports that demonstrated the improved behavior seem to have disappeared |
|
We revisited this during standup today, I think we're going to merge this in and keep an eye on the codecov reports to see if there is any improvement. |
Codecov has been returning weird results for some builds, usually
trending lower (saying that code coverage has dropped by some
ridiculous amount). I think this is because Codecov is getting
confused by all the different reports being uploaded - five from
the PR build, and another five from the branch build.
Here's an example of Codecov conflating reports from different
builds:
https://codecov.io/gh/HumanCellAtlas/data-store/commit/2fcdaa2c27de001e6a436371950d0caecdb40d3d/builds
Closes #2692