Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-60481] Add throttleJobProperty @Symbol to ThrottleJobProperty #68

Merged
merged 2 commits into from
Dec 13, 2019
Merged

[JENKINS-60481] Add throttleJobProperty @Symbol to ThrottleJobProperty #68

merged 2 commits into from
Dec 13, 2019

Conversation

bawjensen
Copy link
Contributor

I pulled inspiration for this change from jenkinsci/branch-api-plugin#127, so I'm hoping it's a fairly straightforward change. In my local testing (using mvn hpi:run and then manually installing Pipeline: Declarative) I can confirm that before this change, the dropdown suggestions for the Declarative Directive Generator for options included only throttle and after this change it included both throttle and throttleJobProperty. When throttleJobProperty was selected the UI that came up was as expected for the job property exposed by this plugin.

Also tested by creating a pipeline job and setting this as its pipeline script:

pipeline {
    agent any

    options {
        throttleJobProperty(
            categories: [],
            limitOneJobWithMatchingParams: false,
            maxConcurrentPerNode: 0,
            maxConcurrentTotal: 4,
            paramsToUseForLimit: '',
            throttleEnabled: true,
            throttleOption: 'project',
        )
    }

    stages {
        stage('build') {
            agent any
            steps {
                sleep 60
                echo "done"
            }
        }
    }
}

Running 5 builds within a minute displayed the expected waiting message, and inspecting the job's configuration showed that it had been persisted as desired there.

Copy link
Member

@oleg-nenashev oleg-nenashev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! It should definitely help users

Copy link
Member

@basil basil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, @bawjensen! I just merged your PR branch with master and applied some trivial formatting fixes. This should be a part of the forthcoming Throttle Concurrent Builds 2.0.2 release.

@basil basil merged commit 5032e15 into jenkinsci:master Dec 13, 2019
@bawjensen
Copy link
Contributor Author

🙇‍♂️ thanks for the quick responses on this, @oleg-nenashev / @basil!

@rory-ye-nv
Copy link

rory-ye-nv commented Jan 21, 2020

when will 2.0.2 will be released, and @bawjensen does this support limitOneJobWithMatchingParams; true for pipeline as well?

if yes, can't wait for this feature to be released.

@rory-ye-nv
Copy link

due to this issue, it's still not work with limitOneJobWithMatchingParams https://issues.jenkins-ci.org/browse/JENKINS-37809

@basil
Copy link
Member

basil commented Feb 19, 2020

when will 2.0.2 will be released

I have just released Throttle Concurrent Builds 2.0.2.

@pandorasbox963
Copy link

pandorasbox963 commented Sep 30, 2020

pipeline {
agent {
node {
label 'slave'
}
}
options {
throttleJobProperty(
categories: ['TEST'],
limitOneJobWithMatchingParams: true,
maxConcurrentPerNode: 1,
maxConcurrentTotal: 2,
paramsToUseForLimit: 'PORT',
throttleEnabled: true,
throttleOption: 'category',
)
}

parameters {
..
}

stages {
..
}

I am using above configuration and I have 2 slave nodes with label slave.
When i run the second build, it is still running on node-1, even though the limit is set to 1 (maxConcurrentPerNode: 1).
Also, i see that paramsToUseForLimit is completely being ignored. It does startup a 2nd build with same parameter value that too on the same node.
What is it that I am missing ? I would greatly appreciate any insights.

@tking16
Copy link

tking16 commented Nov 9, 2020

Same as the above unanswered question. I have 2 nodes with the label 'release-smoke-tests' but the builds are only running one 1. it says in the queue (pending—Already running 1 builds across all nodes) even though the other node is still free!

agent { label 'release-smoke-tests' }
  options {
      throttleJobProperty(
          categories: ['appium-smoke-testing-ios'],
          limitOneJobWithMatchingParams: false,
          maxConcurrentPerNode: 1,
          maxConcurrentTotal: 0,
          throttleEnabled: true,
          throttleOption: 'category',
      )
  }

@edgarkz
Copy link

edgarkz commented Mar 9, 2021

Hello,
Any update on this?

@mandrije
Copy link

mandrije commented Nov 16, 2021

Hello, as explained by @pandorasbox963 above, I'm also unable to limit the number of concurrent executions per node using Throttle Concurrent Builds plugin (ver. 2.2). Pipeline behaves as if maxConcurrentPerNode wasn't set (btw maxConcurrentTotal works as expected). In my case, there are 5 nodes associated with the label I'm using in the pipeline and each one of those has 4 or more executors enabled. NOTE: all these nodes are associated with multiple labels (besides the one I'm targeting).

Is this plugin still maintained?

Update: I just came across this defect... looks like it describes the troublesome behaviour we're highlighting here.

@edgarkz
Copy link

edgarkz commented Nov 16, 2021

Hi @mandrije, we had the same issue with the above plugin, since no resolution was found we switched to https://plugins.jenkins.io/build-blocker-plugin/ instead, and we were able to apply the same restrictions.

@mandrije
Copy link

Thank you very much @edgarkz! Meantime, I realized that the remaining, working options (same param values and max total restrictions) will meet my needs, but hopefully other people reading your suggestion above will benefit from it.

jakubgs added a commit to status-im/nimbus-eth2 that referenced this pull request Jul 25, 2022
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
jenkinsci/throttle-concurrent-builds-plugin#68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/nimbus-eth2 that referenced this pull request Jul 25, 2022
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
jenkinsci/throttle-concurrent-builds-plugin#68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/nimbus-eth2 that referenced this pull request Jul 25, 2022
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
jenkinsci/throttle-concurrent-builds-plugin#68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/nimbus-eth2 that referenced this pull request Jul 26, 2022
A fix for a bug triggered by recent `Jenkinsfile` refactoring done in:
#3827

Which due to a big in Jenkins Throttling plugin caused jobs to start
running in parallel on the same host despite global configuration that
is supposed to block this:
https://issues.jenkins.io/browse/JENKINS-49173
jenkinsci/throttle-concurrent-builds-plugin#68

An attempt to fix this was made in this PR:
#3913

But it was ineffective due to bugs in the Throttle plugin.

As a result semi-random testnet launches would fail with errors like this:
```
./scripts/launch_local_testnet.sh: line 1026: 58977 Killed: 9   ${BEACON_NODE_COMMAND} ...
```
The culprit was the old process cleanup in `scripts/launch_local_testnet.sh`:
```
+ make local-testnet-mainnet
Found old process listening on port 7001, with PID 58977. Killing it.
Found old process listening on port 7002, with PID 59024. Killing it.
Found old process listening on port 7003, withu PID 59027. Killing it.
Found old process listening on port 7004, with PID 59030. Killing it.
```
Which was triggered due to use of immediate assignment for `EXECUTOR_NUMBER`:
```
EXECUTOR_NUMBER := 0
```
Which cause the `EXECUTOR_NUMBER` value set by Jenkins to be ignored.

For more details see:
https://www.gnu.org/software/make/manual/html_node/Flavors.html#Flavors

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/nimbus-eth2 that referenced this pull request Jul 26, 2022
A fix for a bug triggered by recent `Jenkinsfile` refactoring done in:
#3827

Which due to a big in Jenkins Throttling plugin caused jobs to start
running in parallel on the same host despite global configuration that
is supposed to block this:
https://issues.jenkins.io/browse/JENKINS-49173
jenkinsci/throttle-concurrent-builds-plugin#68

An attempt to fix this was made in this PR:
#3913

But it was ineffective due to bugs in the Throttle plugin.

As a result semi-random testnet launches would fail with errors like this:
```
./scripts/launch_local_testnet.sh: line 1026: 58977 Killed: 9   ${BEACON_NODE_COMMAND} ...
```
The culprit was the old process cleanup in `scripts/launch_local_testnet.sh`:
```
+ make local-testnet-mainnet
Found old process listening on port 7001, with PID 58977. Killing it.
Found old process listening on port 7002, with PID 59024. Killing it.
Found old process listening on port 7003, withu PID 59027. Killing it.
Found old process listening on port 7004, with PID 59030. Killing it.
```
Which was triggered due to use of immediate assignment for `EXECUTOR_NUMBER`:
```
EXECUTOR_NUMBER := 0
```
Which cause the `EXECUTOR_NUMBER` value set by Jenkins to be ignored.

For more details see:
https://www.gnu.org/software/make/manual/html_node/Flavors.html#Flavors

Signed-off-by: Jakub Sokołowski <jakub@status.im>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants