Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch validation script to Rucio; rollback py3 changes #10045

Merged
merged 1 commit into from
Nov 5, 2020

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Nov 2, 2020

Fixes #9620

Status

ready

Description

In addition to rolling back some of the python3 modernization changes provided here (which broke the HTTP calls):
#9762

this PR provides the following:

  • checkDsetFileCount.py now compares DBS data against Rucio instead of PhEDEx. It also uses the WMCore/Services modules to talk to the data-services (instead of https/urllib/etc);
  • workflowCompletion.py simply reverted some of the python3 related changes;
  • validate-test-wfs.py replaced PhEDEx by Rucio functionality - without using any of the WMCore libraries, so through their RESTful APIs- and some of the python3 changes have been reverted as well.

Is it backward compatible (if not, which system it affects?)

no

Related PRs

none

External dependencies / deployment changes

none

@amaltaro
Copy link
Contributor Author

amaltaro commented Nov 2, 2020

FYI @todor-ivanov Todor, please fetch this script to validate your integration tests against Rucio.

@amaltaro amaltaro force-pushed the fix-9620 branch 2 times, most recently from 9364469 to 95f8cfd Compare November 2, 2020 21:54
@todor-ivanov
Copy link
Contributor

Thanks @amaltaro I am about to give it a try today.

pylint fixes

do not set Test RSEs to the PNN field

retrieve container level rules too

Remake of dataset file count script to work with Rucio; rollback py3 aliases
@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
  • Pylint check: failed
    • 2 warnings and errors that must be fixed
    • 4 warnings
    • 18 comments to review
  • Pycodestyle check: succeeded
    • 2 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10704/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

amaltaro commented Nov 4, 2020

Given that we are all very busy, I'm not even going to request your review @todor-ivanov and/or @mapellidario ;)
If you feel like looking into it though, please go ahead. Otherwise, only read the main description because it's likely going to be useful for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update scripts and tools to work with Rucio instead of PhEDEx
3 participants