-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tests: Clarify the travis tests using build stages #3150
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3150 +/- ##
==========================================
- Coverage 31.63% 30.85% -0.78%
==========================================
Files 217 217
Lines 6275 6511 +236
Branches 1116 1196 +80
==========================================
+ Hits 1985 2009 +24
- Misses 3606 3744 +138
- Partials 684 758 +74
Continue to review full report at Codecov.
|
406a6c7
to
21bdd31
Compare
.travis.yml
Outdated
stages: | ||
- lint | ||
- unit | ||
- test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no test
stage unless I'm missing something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right, I had a test stage (it's the default stage for the default script) but removed it in the last commit
From what I understand of this feature, I'm not sure we would want to define linting and unit tests as separate stages, since there's no need for them to occur sequentially? |
That's a choice you'd have to make on your own. The idea was to fail early in the process if it's a linting issue, make things clearer IMO but I'm open to using one stage for now. |
21bdd31
to
26039fe
Compare
package.json
Outdated
@@ -126,7 +126,7 @@ | |||
"lint": "eslint .", | |||
"dev": "cross-env BABEL_ENV=default webpack --watch", | |||
"test": "npm run lint && npm run test-unit", | |||
"ci": "concurrently \"npm run lint && npm run build\" \"npm run test-unit:coverage-ci\"", | |||
"ci": "concurrently \"npm run build\" \"npm run test-unit:coverage-ci\"", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to execute npm run build
here, too? It's executed as part of bin/run-wp-unit-tests.sh
for every PHP version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was added later to the PHP unit tests to build the parser. If the parser is no longer built this way in the future (say a Pure PHP parser), this could be removed.
While keeping it here ensure the build is tested as part of the JS unit tests which is more logical to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We moved lint out, so we could do the same with the build, and call npm run test-unit:coverage-ci
directly from Travis. We can leave it as it is, too. I don't think it makes any difference in the end.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand of this feature, I'm not sure we would want to define linting and unit tests as separate stages, since there's no need for them to occur sequentially?
Is it going to be faster to run everything in parallel assuming that all jobs pass? If yes, I would be also in favor of having everything in parallel.
I have no experience with more advanced setup of travis.yml
, but this looks good. It seems like adding another job as a very straightforward task.
Yes, but if linting fails, we'd have lost time. But I understand the point here. Linting don't break often. I just like the logical flow: linting => unit tests => integration tests => deploy Also, there's a limit in concurrent jobs in Travis so the gains are smaller. Though, I'm not sure what's our current limit. Anyway, I'm updating to run everything in parallel |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks solid, all jobs pass. I compared build times with what we had before and I didn't see noticeable changes. You might want to wait for a second review from someone more familiar with Travis :)
Merging, I need to to add job in #3069 |
I like the parallelization. I personally don't see the need for a sequence where linting occurs before unit tests. To your point on wasted time for a failure, I think we ought to optimize for timing of a success case, of which parallelizing should improve (in theory anyways). |
This PR rewrites the
.travis.yml
config file to make use of the new Build Stages feature. This feature has the following pros: