Skip to content
This repository has been archived by the owner on Feb 26, 2024. It is now read-only.

Run separate CI task for all tests with external (apt-get) dependencies #1336

Closed
gnidan opened this issue Oct 12, 2018 · 2 comments · Fixed by #1932
Closed

Run separate CI task for all tests with external (apt-get) dependencies #1336

gnidan opened this issue Oct 12, 2018 · 2 comments · Fixed by #1932
Labels

Comments

@gnidan
Copy link
Contributor

gnidan commented Oct 12, 2018

Issue

Travis is slow and getting slower now that both solc and vyper have to get installed.

Steps to Reproduce

  1. https://travis-ci.org/trufflesuite/truffle
  2. Options -> Trigger Build -> etc.
  3. Wait longer than an amount of time for which it is reasonable to have patience

Expected Behavior

The two main CI jobs (Packages & Integrations) should not run any tests that rely on any packages installed by apt-get.

A separate job should run for specifically these tests. (unknown requirement: should this job be marked as "allowed failure"?)

Background

There are at least two places where CI environment changes how or which tests are run:

  • The external compile tests use the jq utility, so these only run if the CI environment variable is true.

  • A bunch of tests are marked [ @geth ] (e.g.), which gets indicated to mocha by way of the "test filter" option.*

*This approach is annoying because it means the main CI script for running "all the tests" consumes this CLI flag (-k I think but I'm too lazy to look it up). So you can't use -k locally, effectively at all, if you just want to do yarn test -k ...some filter... in the root.

For this issue, it may be worth thinking of a more explicit strategy for how CI manages collections of tests. Or not. Alternate strategy: hack it in and get tests faster by any means necessary.

Environment

Travis-CI

@eggplantzzz
Copy link
Contributor

Random Note: having the keyword 'native' in the string that heads a describe or it block (in the tests) causes that chunk of tests to be excluded while running the tests in truffle-compile due to the testing script found in that package.

@Mordax
Copy link

Mordax commented Oct 23, 2018

Regarding the tests on Vyper failing issue, maybe Vyper can be checked separately to Solidity? Like having two test bundles that depend on a flag to run one at a time when needed, so that any Vyper issues are separated and easier to tackle and build more test support?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants