Bug 1912571: libvirt: support setting network dnsmasq options through the install config #4494

Prashanth684 · 2020-12-18T15:23:09Z

Since libvirt 5.6.0, there is an option to pass in dnsmasq options through the libvirt network [1]. This addresses the following problems:

eliminate the need for hacking routes in the cluster (the workaround mentioned in [3]) so that libvirt's dnsmasq does not manage the domain (and so the requests from inside the cluster will go up the chain to the host itself).
eliminate the hacky workaround used in the multi-arch CI automation to inject *.apps entries in the libvirt network that point to a single worker node [2]. Instead of waiting for the libvirt networks to come up and update entries, we can set this before the installation itself through the install config.
another issue this solves - with the above mentioned workaround, having multiple worker nodes becomes problematic when running upgrade tests. Having the route to just one worker node would fail the upgrade when that worker node is down. With this change, we could now point to the .1 address and have a load balancer forward traffic to any worker node.

With this change, the option can be specified through the install config yaml in the network section as pairs of option name and values. An example:

   platform:
     libvirt:
       network:
         dnsmasqOptions:
         - name: "address"
           value: "/.apps.tt.testing/192.168.126.51"
         if: tt0

The terraform provider supports rendering these options through a datasource and injecting them into the network xml. Since this config is optional, not specifying it will continue to work as before without issues.

[1] https://libvirt.org/formatnetwork.html#elementsNamespaces
[2] https://github.com/openshift/release/blob/master/ci-operator/templates/openshift/installer/cluster-launch-installer-remote-libvirt-e2e.yaml#L532-L554
[3] #1007

jaypoulz · 2020-12-18T16:43:43Z

I was looking into this yesterday when I discovered that 15 of the tests we've disabled depend on dynamic routes. :)
Glad to see you beat me to it. :)

openshift-merge-robot · 2020-12-18T17:21:34Z

@Prashanth684: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-libvirt	`6ec0031`	link	`/test e2e-libvirt`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Prashanth684 · 2020-12-18T17:30:58Z

/test e2e-libvirt

data/data/install.openshift.io_installconfigs.yaml

Prashanth684 · 2020-12-18T18:57:25Z

/retest

EmilienM · 2020-12-18T18:58:51Z

pkg/tfvars/libvirt/libvirt.go

+}
+
+// TFVarsSources contains the parameters to be converted into Terraform variables
+type TFVarsSources struct {


👍 for moving that to a struct, it makes it easily readable.

EmilienM · 2020-12-18T18:59:21Z

small nit wrt naming, otherwise lgtm.

Prashanth684 · 2020-12-18T20:43:16Z

/cc @praveenkumar @cfergeau

Prashanth684 · 2020-12-18T22:08:52Z

/retest

data/data/libvirt/variables-libvirt.tf

cfergeau

Looks good to me apart from a few minor comments. Not very familiar with the terraform plumbing, but since this code has been tested, it's fine :)

data/data/libvirt/main.tf

pkg/types/libvirt/platform.go

data/data/libvirt/variables-libvirt.tf

…b60d7626ff8 This picks up the latest changes in the terraform provider which has support for specifying dnsmasq options

This adds the dnsmasq options block and the corresponding datasource to the libvirt terraform files

openshift-ci-robot · 2021-01-04T19:37:07Z

@Prashanth684: This pull request references Bugzilla bug 1912571, which is invalid:

expected the bug to target the "4.7.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1912571: libvirt: support setting network dnsmasq options through the install config

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2021-02-11T13:43:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T13:56:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T14:09:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T14:48:13Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T15:01:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T15:27:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T15:52:52Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T16:32:02Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T17:23:50Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T17:36:51Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T17:49:19Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T18:15:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T18:28:19Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T18:41:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T19:07:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T19:33:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T20:25:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T20:51:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

yselkowitz · 2021-02-11T21:07:13Z

/bugzilla refresh

openshift-ci-robot · 2021-02-11T21:07:16Z

@yselkowitz: This pull request references Bugzilla bug 1912571, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.8.0) matches configured target release for branch (4.8.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2021-02-11T22:22:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T23:40:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-12T00:58:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci · 2021-02-12T01:15:10Z

@Prashanth684: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-crc	`3c5d034`	link	`/test e2e-crc`
ci/prow/e2e-openstack	`a6ef6c7`	link	`/test e2e-openstack`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2021-02-12T01:50:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2021-02-12T01:52:41Z

@Prashanth684: All pull requests linked via external trackers have merged:

openshift/installer#4494

Bugzilla bug 1912571 has been moved to the MODIFIED state.

In response to this:

Bug 1912571: libvirt: support setting network dnsmasq options through the install config

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from cfergeau and jhixson74 December 18, 2020 15:23

Prashanth684 force-pushed the libvirt-dnsmasq branch 4 times, most recently from bfa3e15 to 6ec0031 Compare December 18, 2020 15:59

EmilienM reviewed Dec 18, 2020

View reviewed changes

data/data/install.openshift.io_installconfigs.yaml Outdated Show resolved Hide resolved

EmilienM reviewed Dec 18, 2020

View reviewed changes

data/data/install.openshift.io_installconfigs.yaml Show resolved Hide resolved

EmilienM reviewed Dec 18, 2020

View reviewed changes

Prashanth684 force-pushed the libvirt-dnsmasq branch from 6ec0031 to 3c5d034 Compare December 18, 2020 19:19

openshift-ci-robot requested a review from praveenkumar December 18, 2020 20:43

Prashanth684 force-pushed the libvirt-dnsmasq branch from 3c5d034 to 9f12426 Compare January 4, 2021 14:30

cfergeau reviewed Jan 4, 2021

View reviewed changes

data/data/libvirt/variables-libvirt.tf Outdated Show resolved Hide resolved

Prashanth684 force-pushed the libvirt-dnsmasq branch from 9f12426 to fd35222 Compare January 4, 2021 16:06

cfergeau approved these changes Jan 4, 2021

View reviewed changes

data/data/libvirt/main.tf Show resolved Hide resolved

pkg/types/libvirt/platform.go Outdated Show resolved Hide resolved

data/data/libvirt/variables-libvirt.tf Show resolved Hide resolved

libvirt: Bump terraform-provider-libvirt to v0.6.4-0.20201216193629-2…

deefdfd

…b60d7626ff8 This picks up the latest changes in the terraform provider which has support for specifying dnsmasq options

Prashanth684 force-pushed the libvirt-dnsmasq branch from fd35222 to 4de091a Compare January 4, 2021 18:39

libvirt: add support for dnsmasq options in the terraform files

791dbf2

This adds the dnsmasq options block and the corresponding datasource to the libvirt terraform files

Prashanth684 force-pushed the libvirt-dnsmasq branch from 4de091a to b60b373 Compare January 4, 2021 18:48

Prashanth684 changed the title ~~libvirt: support setting network dnsmasq options through the install config~~ Bug 1912571: libvirt: support setting network dnsmasq options through the install config Jan 4, 2021

openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jan 4, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 11, 2021

openshift-merge-robot merged commit 6b89cd1 into openshift:master Feb 12, 2021

Prashanth684 deleted the libvirt-dnsmasq branch February 12, 2021 14:13

deepsm007 mentioned this pull request Feb 22, 2021

Adding the dnsmasq changes to the template openshift/release#16135

Merged

Bug 1912571: libvirt: support setting network dnsmasq options through the install config #4494

Bug 1912571: libvirt: support setting network dnsmasq options through the install config #4494

Conversation

Prashanth684 commented Dec 18, 2020 • edited Loading

jaypoulz commented Dec 18, 2020

openshift-merge-robot commented Dec 18, 2020

Prashanth684 commented Dec 18, 2020

Prashanth684 commented Dec 18, 2020

EmilienM Dec 18, 2020

Choose a reason for hiding this comment

EmilienM commented Dec 18, 2020

Prashanth684 commented Dec 18, 2020

Prashanth684 commented Dec 18, 2020

cfergeau left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Jan 4, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

yselkowitz commented Feb 11, 2021

openshift-ci-robot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 11, 2021

openshift-bot commented Feb 12, 2021

openshift-ci bot commented Feb 12, 2021 • edited Loading

openshift-bot commented Feb 12, 2021

openshift-ci-robot commented Feb 12, 2021

Prashanth684 commented Dec 18, 2020 •

edited

Loading

openshift-ci bot commented Feb 12, 2021 •

edited

Loading