The CI is too slow

[decentral1se]

Issue Type

• Gripe about the CI

On https://travis-ci.com/ansible/molecule/builds/97459293, we saw:

Ran for 1 hr 26 min 9 sec
Total time 4 hrs 59 min 45 sec 

Hopefully this ticket can be a place to gather ideas on how to speed this process up.

PS. I am thankful for all the great work that has been done to get this CI in place.

[ragingpastry]

Is this something that could be fixed using a tool like detox?

[ssbarnea]

I am not sure if there is much to be done about it as molecule needs to test a huge combination of python versions, ansible versions, platforms/distributions, containers/vms/clouds...

Even with a paid travis plan this could be limited. I wonder if using openstack build infrastructure would not be a better way to test changes.

[ragingpastry]

Detox would help the execution of python / ansible tests as they would happen concurrently which could speed some things up. Maybe "fixed" was the incorrect word to use but I could see concurrent builds working fairly quickly.

[decentral1se]

OK, detox might be a good first step then but I see in that repository link:

detox is unmaintained and incompatible with tox > 3.6. detox was a plugin for tox to enable parallel environment execution. tox 3.7 added a native possibility to do this (tox -p|--parallel) and effectively supercedes detox.

So, perhaps we can improve by simply passing -p FOO!

EDIT: See https://tox.readthedocs.io/en/latest/config.html.

[decentral1se]

OK, after some experimenting. I see that we can't get anything out of detox/tox -p because we run each build environment in isolation. We can't parallelize this. I thought it might be useful in that it would speed up somehow these individual builds (by using multiple cores) but that functionality doesn't seem to offered.

[themr0c]

Would sitepackages = true in tox.ini bring some improvement?

[themr0c]

Some tests don't need the all armada of requirements to be installed, so we could gain time here also. For instance, AFAIK, lint only need flake8 (and should not depend on, let's say, having lxc up and running)

[decentral1se]

@themr0c, good idea about the requirements! Not sure about sitepackages. Let's try these.

And just to link, #1698 looks like it will definitely help!

[decentral1se]

#1695 addresses the linting install skips? Think so.

[webknjaz]

Related to: #1702 (comment)

Once the problem with parallel Molecule execution is solved, we can try using thread-based parallelism which via pytest-xdist. It could improve things dramatically.

[decentral1se]

dramatically

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

[themr0c]

Are we not installing the packages twice ? First time with deps = -rrequirements.txt, and a second time with PBR install (that can be skipped with skip_install = true) ? So maybe removing the line with rrequirements.txt could help gain some seconds ?

[webknjaz]

@themr0c I hope once pbr is removed, we'll have it in one place anyway. I'd not touch it right now.

[decentral1se]

I think we've done a lot of good things with regard to this. The major remaining issue (run things in parallel) is tracked elsewhere and we're also moving forward with Zuul integration (see #1770) too! I'm gonna close this off.