Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to GPUs in TaskChain spec #10805

Merged
merged 2 commits into from
Sep 14, 2021
Merged

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Sep 14, 2021

Fixes #10400

Status

ready

Description

As a follow up of: #10799

this PR enabled GPU support in the TaskChain spec, thus, supporting it inside each Task definition as well.

Note that for TaskChain workflows, there is the same default values as those for top level, thus:

  • RequiresGPU : forbidden
  • GPUParams: json.dumps(None)
    and the top level arguments will have
  • RequiresGPU: None

Is it backward compatible (if not, which system it affects?)

YES (new arguments were made optional)

Related PRs

Complement to: #10799

External dependencies / deployment changes

None

@cmsdmwmbot
Copy link

Jenkins results:

  • Python2 Unit tests: succeeded
    • 1 tests added
  • Python3 Unit tests: succeeded
    • 1 tests added
    • 2 changes in unstable tests
  • Python2 Pylint check: failed
    • 22 warnings and errors that must be fixed
    • 4 warnings
    • 50 comments to review
  • Python3 Pylint check: failed
    • 29 warnings and errors that must be fixed
    • 4 warnings
    • 69 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 13 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12442/artifact/artifacts/PullRequestReport.html

validate top level GPU in TaskChain
TaskChain unit tests for GPU
@cmsdmwmbot
Copy link

Jenkins results:

  • Python2 Unit tests: succeeded
    • 3 tests added
  • Python3 Unit tests: succeeded
    • 3 tests added
    • 1 changes in unstable tests
  • Python2 Pylint check: failed
    • 30 warnings and errors that must be fixed
    • 5 warnings
    • 88 comments to review
  • Python3 Pylint check: failed
    • 37 warnings and errors that must be fixed
    • 5 warnings
    • 116 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 72 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12444/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

Basic tests went fine in my VM. @todor-ivanov feel free to leave any questions and/or your review. I will tackle the StepChain use case tomorrow, so I can fix whatever is left from this one. Thanks!

@amaltaro amaltaro merged commit a30bd3f into dmwm:master Sep 14, 2021
@@ -1217,6 +1217,9 @@ def getChainCreateArgs(firstTask=False, generator=False):
'Memory': {'default': None, 'null': True, 'type': float, 'validate': lambda x: x > 0},
'Multicore': {'default': 0, 'type': int, 'validate': lambda x: x > 0},
'PrepID': {'default': None, 'null': True, 'optional': True, 'type': str},
"RequiresGPU": {"default": "forbidden",
Copy link
Contributor

@todor-ivanov todor-ivanov Sep 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope these few lines do not overlap with the some of the previous PR. Otherwise this may cause some conflicts.

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me @amaltaro
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for GPU parameters at TaskChain spec level
3 participants