Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten all rpm packages other than atomic-openshift-utils #4264

Closed
wants to merge 1 commit into from

Conversation

sdodson
Copy link
Member

@sdodson sdodson commented May 23, 2017

I can think of no situation where someone would want to individually install roles, playbooks, filter_plugins, or other subpackages without installing all the rest. The same applies to upgrading each of those. So in the interest of simplicity lets squash the packaging back down to just the quick installer (atomic-openshift-utils) and ansible code (openshift-ansible).

Fixes #3469

@sdodson
Copy link
Member Author

sdodson commented May 23, 2017

[test]

@sdodson
Copy link
Member Author

sdodson commented May 23, 2017

@stevekuznetsov @jhadvig FYI

@stevekuznetsov
Copy link
Contributor

What do we need to install now? This looks like it will be backwards incompatible with the current jobs but we can hack around it so it can go in more cleanly.

@sdodson
Copy link
Member Author

sdodson commented May 23, 2017

@stevekuznetsov The provides and obsoletes should make it transparent but long term you'll just want to drop everything other than openshift-ansible.

@sdodson
Copy link
Member Author

sdodson commented May 23, 2017

The previous test job makes it look like obsoletes means you cannot install a specific version of a package. I'll try some more testing locally.

@sdodson sdodson changed the title Flatten all rpm packages other than atomic-openshift-utils [WIP] Flatten all rpm packages other than atomic-openshift-utils May 23, 2017
@stevekuznetsov
Copy link
Contributor

stevekuznetsov commented May 23, 2017

We can also add glue logic to the jobs for now:

if yum list available atomic-openshift-utils-${version} >/dev/null 2>&1; then
    yum install atomic-openshift-utils-${version}
else
    yum install openshift-ansible-${version}
fi

@openshift-bot openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 23, 2017
@sdodson sdodson force-pushed the rpm-refactor branch 2 times, most recently from 2630b04 to 263d2f4 Compare August 10, 2017 20:31
@sdodson
Copy link
Member Author

sdodson commented Aug 10, 2017

[test]

@openshift-bot openshift-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 10, 2017
@sdodson sdodson changed the title [WIP] Flatten all rpm packages other than atomic-openshift-utils Flatten all rpm packages other than atomic-openshift-utils Aug 28, 2017
@sdodson
Copy link
Member Author

sdodson commented Aug 28, 2017

It seems to cleanly upgrade now. So this is ready for review.

# rpm -qa | grep openshift-ansible
openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-roles-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-docs-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-lookup-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-callback-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-playbooks-3.6.173.0.5-3.git.0.522a92a.el7.noarch
openshift-ansible-filter-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch
[root@ose3-master ~]# yum install ./openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch.rpm 
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Examining ./openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch.rpm: openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch
Marking ./openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch.rpm as an update to openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch
Resolving Dependencies
--> Running transaction check
---> Package openshift-ansible.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be updated
---> Package openshift-ansible.noarch 0:3.7.1-1.git.163.263d2f4.el7 will be obsoleting
---> Package openshift-ansible-callback-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
---> Package openshift-ansible-docs.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
---> Package openshift-ansible-filter-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
---> Package openshift-ansible-lookup-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
---> Package openshift-ansible-playbooks.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
---> Package openshift-ansible-roles.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7 will be obsoleted
--> Finished Dependency Resolution

Dependencies Resolved

==================================================================================================================================================================================================================================================================================
 Package                                                   Arch                                           Version                                                             Repository                                                                                     Size
==================================================================================================================================================================================================================================================================================
Installing:
 openshift-ansible                                         noarch                                         3.7.1-1.git.163.263d2f4.el7                                         /openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch                                          28 M
     replacing  openshift-ansible-callback-plugins.noarch 3.6.173.0.5-3.git.0.522a92a.el7
     replacing  openshift-ansible-docs.noarch 3.6.173.0.5-3.git.0.522a92a.el7
     replacing  openshift-ansible-filter-plugins.noarch 3.6.173.0.5-3.git.0.522a92a.el7
     replacing  openshift-ansible-lookup-plugins.noarch 3.6.173.0.5-3.git.0.522a92a.el7
     replacing  openshift-ansible-playbooks.noarch 3.6.173.0.5-3.git.0.522a92a.el7
     replacing  openshift-ansible-roles.noarch 3.6.173.0.5-3.git.0.522a92a.el7

Transaction Summary
==================================================================================================================================================================================================================================================================================
Install  1 Package

Total size: 28 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch                                                                                                                                                                                                           1/8 
  Erasing    : openshift-ansible-playbooks-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                             2/8 
  Erasing    : openshift-ansible-roles-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                 3/8 
  Erasing    : openshift-ansible-callback-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                      4/8 
  Erasing    : openshift-ansible-filter-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                        5/8 
  Erasing    : openshift-ansible-lookup-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                        6/8 
  Erasing    : openshift-ansible-docs-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                  7/8 
  Cleanup    : openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                       8/8 
  Verifying  : openshift-ansible-3.7.1-1.git.163.263d2f4.el7.noarch                                                                                                                                                                                                           1/8 
  Verifying  : openshift-ansible-lookup-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                        2/8 
  Verifying  : openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                       3/8 
  Verifying  : openshift-ansible-filter-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                        4/8 
  Verifying  : openshift-ansible-callback-plugins-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                      5/8 
  Verifying  : openshift-ansible-playbooks-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                             6/8 
  Verifying  : openshift-ansible-docs-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                  7/8 
  Verifying  : openshift-ansible-roles-3.6.173.0.5-3.git.0.522a92a.el7.noarch                                                                                                                                                                                                 8/8 

Installed:
  openshift-ansible.noarch 0:3.7.1-1.git.163.263d2f4.el7                                                                                                                                                                                                                          

Replaced:
  openshift-ansible-callback-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                   openshift-ansible-docs.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                        openshift-ansible-filter-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                  
  openshift-ansible-lookup-plugins.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                     openshift-ansible-playbooks.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                   openshift-ansible-roles.noarch 0:3.6.173.0.5-3.git.0.522a92a.el7                           

Complete!

@sdodson
Copy link
Member Author

sdodson commented Aug 28, 2017

Ok, I'd like to merge this soon.
@smunilla FYI as packages will be dropped, i can reach out to RCM but wanted to make sure you know
@jhadvig FYI for you too, obsoletes and provides should make this transparent

@sdodson
Copy link
Member Author

sdodson commented Aug 28, 2017

Got rid of the whitespace diffs.

@sdodson
Copy link
Member Author

sdodson commented Aug 28, 2017

aos-ci-test

@openshift-bot
Copy link

success: "aos-ci-jenkins/OS_3.6_containerized, aos-ci-jenkins/OS_3.6_containerized_e2e_tests" for 08c758f (logs)

@openshift-bot
Copy link

success: "aos-ci-jenkins/OS_3.6_NOT_containerized, aos-ci-jenkins/OS_3.6_NOT_containerized_e2e_tests" for 08c758f (logs)

@sdodson
Copy link
Member Author

sdodson commented Aug 30, 2017

[merge]

@sosiouxme
Copy link
Member

sosiouxme commented Aug 31, 2017

@sdodson did the bot miss this? what if i say merge ...

Edit: rosie was ignoring this PR because it's old.

@ingvagabund
Copy link
Member

[merge]

@openshift-bot
Copy link

Evaluated for openshift ansible test up to 08c758f

@openshift-bot
Copy link

continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_openshift_ansible/578/) (Base Commit: a00df57) (PR Branch Commit: 08c758f)

@sosiouxme
Copy link
Member

https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_openshift_ansible/959/ has flakes openshift/origin#16005 and openshift/origin#10162
-- I don't understand what happened with RUN INTEGRATION TESTS, doesn't look like they really ran at all, not sure if it was oom_killed or what happened.

Try another [merge]

@openshift-bot
Copy link

Evaluated for openshift ansible merge up to 08c758f

@openshift-bot
Copy link

continuous-integration/openshift-jenkins/merge FAILURE (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_openshift_ansible/961/) (Base Commit: a00df57) (PR Branch Commit: 08c758f)

@sosiouxme
Copy link
Member

sosiouxme commented Sep 1, 2017

https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_openshift_ansible/961/ looks like a legit failure. It seems the obsoletes are messing things up when yum tries to set up for the upgrade.

+ echo '=== Installing atomic-openshift-utils-3.6.173.0.27-1.git.0.be1701e.el7 packages ==='
+ sudo yum install -y openshift-ansible-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-callback-plugins-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-docs-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-filter-plugins-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-lookup-plugins-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-playbooks-3.6.173.0.27-1.git.0.be1701e.el7 openshift-ansible-roles-3.6.173.0.27-1.git.0.be1701e.el7
Loaded plugins: amazon-id, rhui-lb, search-disabled-repos
Package openshift-ansible-callback-plugins is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Package openshift-ansible-docs is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Package openshift-ansible-filter-plugins is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Package openshift-ansible-lookup-plugins is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Package openshift-ansible-playbooks is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Package openshift-ansible-roles is obsoleted by openshift-ansible, trying to install openshift-ansible-3.7.1-1.git.0.b9cc810.el7.noarch instead
Resolving Dependencies
--> Running transaction check
---> Package openshift-ansible.noarch 0:3.6.173.0.27-1.git.0.be1701e.el7 will be installed
--> Processing Dependency: python-passlib for package: openshift-ansible-3.6.173.0.27-1.git.0.be1701e.el7.noarch
--> Processing Dependency: httpd-tools for package: openshift-ansible-3.6.173.0.27-1.git.0.be1701e.el7.noarch
---> Package openshift-ansible.noarch 0:3.7.1-1.git.0.b9cc810.el7 will be installed

It seems like even though it specifically requests the 3.6 versions, the obsoletes from the 3.7 version are taking effect and making yum try to install that instead of the obsolete RPMs. This seems weird to me. I'm not sure this would be a problem in any customer use case but it obviously is in CI and probably would be for online too.

@sdodson
Copy link
Member Author

sdodson commented Sep 5, 2017

Testing with Obsoletes >= 3.7

@sdodson
Copy link
Member Author

sdodson commented Sep 6, 2017

I've tried removing the Provides and it fails with a depsolving issue. @jhadvig Do we have the option of making sure that 3.7 repos aren't enabled when 3.6 packages are being installed?

@sosiouxme
Copy link
Member

@jhadvig @sdodson any ideas for moving this along? maybe an excluder could prevent the 3.7 RPMs from being available until the upgrade?

Copy link
Member

@jhadvig jhadvig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my perspective it makes sense to flatten the pkgs. But we will have to make sure that we either backport this change to all the versions of openshift-ansible or make sure that we condition our tests to use the openshift-ansible-utils pkg from the version that will be build now on.

@michaelgugino
Copy link
Contributor

@sdodson Are we going to ship this?

@sdodson
Copy link
Member Author

sdodson commented Oct 26, 2017

@sdodson Are we going to ship this?

one day we will, it works fine with the exception of some rather broken CI processes.

@openshift-merge-robot
Copy link
Contributor

@sdodson PR needs rebase

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 28, 2017
@sdodson sdodson mentioned this pull request Dec 19, 2017
@michaelgugino
Copy link
Contributor

@sdodson let's ship this puppy.

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 11, 2018
@ingvagabund
Copy link
Member

@sdodson is this PR still relevant?

@openshift-ci-robot
Copy link

@sdodson: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/openshift-jenkins/tox 08c758f link /test tox
ci/openshift-jenkins/gcp 08c758f link /test gcp
ci/openshift-jenkins/gcp-upgrade 08c758f link /test gcp-upgrade
ci/openshift-jenkins/install 08c758f link /test install
ci/prow/e2e-aws 08c758f link /test e2e-aws

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@vrutkovs
Copy link
Member

Master branch is closed! A major refactor is ongoing in devel-40. Changes for 3.x should be made directly to the latest release branch they're relevant to and backported from there.

@vrutkovs vrutkovs closed this Dec 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants