Performance issues with lots of parallel steps

[victorb]

I'm trying to parallize more steps, but quickly hit some bottleneck in jenkins where the more parallel steps you have in a build, the slower every step becomes. So rather than taking ~1 seconds to check if a file exists in a step, it now takes 30 seconds, which is unacceptable and unworkable if we want to parallize tasks. Even steps like withEnv, which should be sub-second, now takes multiple seconds.

@magik6k could you help me debug this? I know you have some dark magic powers when it comes to finding bottlenecks!

Found two existing issues in the Jenkins issue tracker:

• https://issues.jenkins-ci.org/browse/JENKINS-47724
• https://issues.jenkins-ci.org/browse/JENKINS-45876

[victorb]

This issue was moved to ipfs-inactive/dev-team-enablement#113