package openshift-ansible-3.6.144-1.git.0.1382071.el7 is not installed flake

[0xmichalis]

+ rpm -V openshift-ansible-3.6.144-1.git.0.1382071.el7
package openshift-ansible-3.6.144-1.git.0.1382071.el7 is not installed
++ export status=FAILURE
++ status=FAILURE
+ set +o xtrace
########## FINISHED STAGE: FAILURE: VERIFY THE INSTALLED OPENSHIFT-ANSIBLE RELEASE [00h 00m 00s] ##########

https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_extended_conformance_install_update/2345/consoleFull#128703821658b6e51eb7608a5981914356

@sdodson

[sdodson]

@jhadvig

You install 3.5.96 and then verify that 3.6.144 is there.

+ sudo yum upgrade -y openshift-ansible-3.5.96-1.git.0.88536f9.el7 openshift-ansible-callback-plugins-3.5.96-1.git.0.88536f9.el7 openshift-ansible-docs-3.5.96-1.git.0.88536f9.el7 openshift-ansible-filter-plugins-3.5.96-1.git.0.88536f9.el7 openshift-ansible-lookup-plugins-3.5.96-1.git.0.88536f9.el7 openshift-ansible-playbooks-3.5.96-1.git.0.88536f9.el7 openshift-ansible-roles-3.5.96-1.git.0.88536f9.el7
...
...
+ rpm -V openshift-ansible-3.6.144-1.git.0.1382071.el7
package openshift-ansible-3.6.144-1.git.0.1382071.el7 is not installed

[jhadvig]

So the issue here is that when the OPENSHIFT_ANSIBLE_TARGET_BRANCH is pointing to any of release-1.x then we are installing x-1 version of the openshift-ansible pkgs and upgrading to x version. But script responsible for the verification is pointing to the PR's head which in this case is different minor version

cd "\${GOPATH}/src/github.com/openshift/openshift-ansible"
last_tag="\$( git describe --tags --abbrev=0 --exact-match HEAD )"
last_commit="\$( git log -n 1 --pretty=%h )"

@stevekuznetsov fyi

[stevekuznetsov]

@jhadvig I don't understand. The check is that we have correctly upgraded to the version we just built from whatever code is sitting in openshift-ansible source. We need to upgrade to that version, otherwise what are we testing?

[stevekuznetsov]

But script responsible for the verification is pointing to the PR's head

Script is looking at the merge of master and PR

[stevekuznetsov]

Here's what we are building in the job:

########## STARTING STAGE: BUILD AN OPENSHIFT-ANSIBLE RELEASE ##########
...
+ tito tag --offline --accept-auto-changelog
Creating output directory: /tmp/tito
Tagging new version of openshift-ansible: 3.6.143-1 -> 3.6.144-1

The point of the check I added is to catch cases like this where we are not installing the version of the product we just built.

[jhadvig]

@stevekuznetsov in this particular case the PR is backported to the 1.5 origin branch, and therefor we are:

1. building the origin-1.5.x
2. installing openshift-ansible-1.4.x
3. installing origin-1.4.x
4. upgrading openshift-ansible to 1.5.x
5. upgrading origin to the built 1.5.x

In this case there is no point to build openshift-ansible pkgs cause we need to install openshift-ansible in corresponding versions, in this case install 1.4.x and upgrade to 1.5.x

If you check the test output you will see that its building the latest version of the openshift-ansible

+ tito tag --offline --accept-auto-changelog
Creating output directory: /tmp/tito
Tagging new version of openshift-ansible: 3.6.143-1 -> 3.6.144-1

Created tag: openshift-ansible-3.6.144-1
   View: git show HEAD
   Undo: tito tag -u
   Push: git push origin && git push origin openshift-ansible-3.6.144-1

[jhadvig]

What would be the point of building openshift-ansible-1.4.x && 1.5.x in this particular case ?

[stevekuznetsov]

In this case there is no point to build openshift-ansible pkgs

We should a) be in the right o-a branch and b) be building the latest version of that o-a

[0xmichalis]

https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_extended_conformance_install_update/2366/consoleFull#128703821658b6e51eb7608a5981914356

[jhadvig]

We should a) be in the right o-a branch and b) be building the latest version of that o-a

But why we need to build the o-a in this particular case ? If anything I would check for the ORIGIN_TARGET_BRANCH when building o-a and if its pointing to a release branch I would skip the build of o-a and just install corresponding version that already built and in paas sig repo (and vice versa with the OPENSHIFT_ANSIBLE_TARGET_BRANCH and origin). That would also help with the issue of reducing the time for the install_upgrade job

[jhadvig]

We are already skipping the building of origin when OPENSHIFT_ANSIBLE_TARGET_BRANCH is specified https://github.com/openshift/aos-cd-jobs/blob/master/sjb/config/test_cases/test_branch_origin_extended_conformance_install_update.yml#L77-L87

[jhadvig]

But if you insist on building the o-a in the matching branch I've created a fix will checkout appropriate branch and build o-s -> jhadvig/aos-cd-jobs@378077d

Running tests with mentioned changes:

• PR against release 1.5 - https://ci.openshift.redhat.com/jenkins/view/All/job/jhadvig/5/
• PR against master - https://ci.openshift.redhat.com/jenkins/view/All/job/jhadvig/6/

[stevekuznetsov]

That is the point of this job -- build latest o-a and origin, test them together. We want this job to be a gate for us releasing versions of both of these projects. The covenant for the job is it tells us about the latest code in both repos (as appropriate).