Rebase 1.11 #20033

deads2k · 2018-06-18T16:49:13Z

cmd/hypershift builds, nothing else. I'm only opening it now so that I can point out how much we'd gain by splitting separate components out of the core repo.

Failing

gcp - red, certs

Passing

conformance - green
extended builds - green
image-registry - green, flaking
cross-build - green
unit - green - flaking, flakes noted below
integration - green
cluster-up green
service-catalog green
test-cmd - green
test-end-to-end: green
verify - green

unit flakes:

github.com/openshift/origin/pkg/dns Test_dnsmasqMonitor_run_metrics 0.01s
dnsmasq_test.go:180: got an unexpected second call: uk.org.thekelleys.SetDomainServers
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubelet/util/pluginwatcher - vendor/k8s.io/kubernetes/pkg/kubelet/util/pluginwatcher flake #20147
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/cloudprovider/providers/gce TestEnsureInternalBackendServiceGroups

deads2k · 2018-06-18T16:50:25Z

glide.yaml

 # pod - sjenning
+# origin-3.11-cadvisor-v0.30.1 MISSING PATCHES


@sjenning patches do not apply. This is going to backlevel you. You'll need to update the fork.

fork has been updated openshift/google-cadvisor#3 @runcom

deads2k · 2018-06-18T16:51:00Z

pkg/oauthserver/oauth/handlers/default_auth_handler.go

-					if err := authHandler.requestContextMapper.Update(req, ctx); err != nil {
-						return false, fmt.Errorf("failed to attach audit event to context: %v", err)
-					}
+					req = req.WithContext(request.WithAuditEvent(ctx, ev))


@liggitt did I get this bit right?

I actually don't see why the requestContextMapper.Update call was required before... we're updating a field of a pointer audit event in place. I don't see why the ev.ResponseStatus.Message = getAuthMethods(req) call wouldn't be all that is required.

the new request isn't being propagated anywhere useful, so the req = req.WithContext(...) isn't likely to be effective.

@soltysh, where is this data supposed to end up, and how do we tell if it is correct?

The idea behind it is to log the failed authn requests, since we can't pass the failedHandler, like uptream does. This will be then logged in the regular audit log. The way it was previously done was to update the request with the new data in the context. I don't think the current implementation is right, you're just updating the request object, but you're not updating the mapper between request and context with the new state. The other option is that my understanding of requestContextMapper is wrong, in which case this is not needed at all.

I think req = req.WithContext(request.WithAuditEvent(ctx, ev)) should be dropped

@soltysh, can you describe what we should look for in an audit event to know if this is working correctly with that removed? a failed oauth login attempt should result in an audit event with the auth method included?

WithAuditEvent just wraps the context (Golang's linear wrapping chain context design...). So the returned context is a different object than before. You have to attach that to the request again. That's why we had the update.

Same is true for the req = req.WithContext(...) line now. If you look into WithContext, it does a shallow copy, but does not mutate the old req.

Or in other words: a context is not a map which is updated in-place. It is a chain of if key == bla { .... } else ask-the-underlying context constructs.

Right, but we're modifying the existing audit event here, so we shouldn't need to change the context at all

Yep, AuditEventFrom is called before. So we change the event in place. Didn't see that from the snippet.

deads2k · 2018-06-18T17:48:46Z

@jwmatthews I think this is the first time you're in the rebase. There's a commit with JMATTHEWS in the title. You don't need to worry about the rest of the pull, but you should review that commit for accuracy. In this case, it's related to the TSB.

mfojtik · 2018-06-19T08:37:13Z

pkg/cmd/server/origin/admission/plugin_initializer.go

@@ -112,26 +108,22 @@ func NewPluginInitializer(
 	svcCache := service.NewServiceResolverCache(kubeInternalClient.Core().Services(metav1.NamespaceDefault).Get)
 	defaultRegistryFunc, err := svcCache.Defer(defaultRegistry)
 	if err != nil {
-		return nil, nil, fmt.Errorf("OPENSHIFT_DEFAULT_REGISTRY variable is invalid %q: %v", defaultRegistry, err)
+		return nil, fmt.Errorf("OPENSHIFT_DEFAULT_REGISTRY variable is invalid %q: %v", defaultRegistry, err)


reminder that we should nuke this environment variable at this point

mfojtik · 2018-06-19T08:37:49Z

pkg/cmd/server/origin/master_config.go


 	"github.com/golang/glog"
 	"github.com/openshift/origin/pkg/admission/namespaceconditions"
+	"k8s.io/apimachinery/pkg/api/meta"


nit: sort the imports based on conventions

mfojtik · 2018-06-19T08:38:43Z

pkg/cmd/server/origin/master_config.go

@@ -199,7 +206,15 @@ func BuildMasterConfig(

 		kubeAPIServerConfig: kubeAPIServerConfig,
 		additionalPostStartHooks: map[string]genericapiserver.PostStartHookFunc{
-			"openshift.io-AdmissionInit": admissionPostStartHook,
+			"openshift.io-RESTMapper": func(context genericapiserver.PostStartHookContext) error {


maybe post-rebase but can we extract this into separate function?

mfojtik · 2018-06-19T08:38:58Z

pkg/cmd/server/origin/openshift_apiserver.go

@@ -9,6 +9,7 @@ import (

 	restful "github.com/emicklei/go-restful"
 	"github.com/golang/glog"
+	"k8s.io/apimachinery/pkg/api/meta"


nit: add empty line...

mfojtik · 2018-06-19T08:40:04Z

pkg/project/apiserver/apiserver.go

@@ -4,6 +4,8 @@ import (
 	"sync"

 	"github.com/golang/glog"
+	"k8s.io/apimachinery/pkg/api/meta"


nit: guess your goimports is broken

mfojtik · 2018-06-19T08:41:14Z

pkg/project/registry/projectrequest/delegated/delegated.go

-		if gvk := info.Mapping.GroupVersionKind; kapierror.IsAlreadyExists(err) &&
-			gvk.Kind == roleBindingKind && roleBindingGroups.Has(gvk.Group) && defaultRoleBindingNames.Has(info.Name) {
-			return false
+	// TODO, stop doing this crazy thing, but for now it's a very simple way to get the unstructured objects we need


can you add this into post-rebase issues?

mfojtik · 2018-06-19T08:46:51Z

pkg/apps/registry/deploylog/rest.go

 	}
 	return firstPod.Name, nil
 }
+
+// GetFirstPod returns a pod matching the namespace and label selector


@deads2k this is the stuff we need to to copy out from kubectl ? can we have follow up issue where we extract this from kubectl into some kind of library in kube?

(also if yes, can you add comment/todo that this was copied out? )

mfojtik · 2018-06-19T08:48:20Z

pkg/apps/registry/deploylog/rest.go

+	if err != nil {
+		return nil, 0, err
+	}
+	pods := []*corev1.Pod{}


can we initialize this with make and len(podList.Items) ?

actually nvmd. this should be fixed upstream...

mfojtik · 2018-06-19T08:52:06Z

@deads2k commits with my name lgtm, but I have question about e52ebc1 ... can you explain me why this is needed in the rebase ?

soltysh · 2018-06-19T12:58:24Z

SOLTYSH commits lgtm, so far.

deads2k · 2018-06-19T20:48:57Z

Reminder to self not to squash anything for mfojtik or soltysh before JMATTHEWS: avoid kubectl dependency in template service broker yet

deads2k · 2018-06-19T21:02:59Z

the branch builds (but not on master apparently). Oh the conflicts.

@soltysh are the oc changes critical? Can we hold some oc changes out? those may be mine I guess....

deads2k · 2018-06-19T21:11:03Z

@soltysh, it compiles. Barring major problems, I don't plan to clean up oc anymore.

deads2k · 2018-06-19T21:12:51Z

Also, I started switching printers, but it was too much. I may revert the printer commit and keep the patch to get through this rebase if I start having trouble

liggitt · 2018-06-19T21:39:02Z

pkg/cmd/server/start/start_node.go

@@ -489,7 +489,7 @@ func StartNode(nodeConfig configapi.NodeConfig, components *utilflags.ComponentF

 // runKubeletInProcess runs the kubelet command using the provide args
 func runKubeletInProcess(kubeletArgs []string) error {
-	cmd := kubeletapp.NewKubeletCommand()
+	cmd := kubeletapp.NewKubeletCommand(wait.NeverStop)


follow up to plumb the channel that gets notified of interrupt signals

liggitt · 2018-06-19T21:45:57Z

pkg/bulk/cmd.go

-		if len(resource) > 0 {
-			fmt.Fprintf(out, "%s/%s\n", resource, name)
+		if len(kindString) > 0 {
+			fmt.Fprintf(out, "%s/%s\n", kindString, name)


lowercased, right?

liggitt · 2018-06-19T21:46:10Z

pkg/bulk/cmd.go

-		if len(resource) > 0 {
-			fmt.Fprintf(out, "%s \"%s\" %s%s\n", resource, name, operation, dryRunMsg)
+		if len(kindString) > 0 {
+			fmt.Fprintf(out, "%s \"%s\" %s%s\n", kindString, name, operation, dryRunMsg)


lowercased, right?

liggitt · 2018-06-19T21:48:34Z

pkg/security/securitycontextconstraints/provider.go

+	return sysctl.NewMustMatchPatterns(safeWhitelist, allowedUnsafeSysctls, forbiddenSysctls), nil
+}
+
+// TODO promote like kube did


/cc @ingvagabund

/cc @php-coder @stlaz

…broken

Underlying gonum/graph assigns conflicting IDs to new nodes after node deletion. Thus two different nodes amy be assigned the same ID. This leads to panic when an edge is set between them. Temporarily disabling event handling until the package is fixed. This ensures that no new nodes will be added to the graph after the first node gets removed. Signed-off-by: Michal Minář <miminar@redhat.com>

soltysh · 2018-06-28T11:33:02Z

pkg/oc/admin/prune/authprune/group_test.go

@@ -138,10 +134,12 @@ func TestGroupReaper(t *testing.T) {

 		actual := []interface{}{}
 		oreactor := func(action clientgotesting.Action) (handled bool, ret runtime.Object, err error) {
+			t.Logf("oreactor: %#v", action)


Nit: my bad, that is not needed here. Post-rebase remove (note to myself).

deads2k · 2018-06-28T11:41:40Z

/test launch-gcp

deads2k · 2018-06-28T12:12:07Z

@miminar looks like the image pruning fix turned a test red. I need to let this run for GCE but on the next push I'm going to disable that test.

deads2k · 2018-06-28T12:12:32Z

verify is green when you look at the link, but showing red here.

/retest

miminar · 2018-06-28T12:19:10Z

@deads2k yeah, it's TestChangeImageStreamsWhilePruning. Sorry, I should have made it myself.

deads2k · 2018-06-28T15:28:40Z

new unit flake: https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/20033/pull-ci-origin-unit/535/

/retest

deads2k · 2018-06-28T15:36:41Z

https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/20033/pull-ci-origin-e2e-gcp/135/

/retest

deads2k · 2018-06-28T16:26:20Z

/test unit

…ses when we got none from the cloudprovider" This reverts commit 7354bbe.

mfojtik · 2018-06-28T17:14:45Z

/lgtm

openshift-ci-robot · 2018-06-28T17:14:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, mfojtik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [deads2k,mfojtik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sjenning · 2018-06-28T17:20:10Z

we are back to certificate signed by unknown authority in the e2e logs

deads2k · 2018-06-28T17:45:15Z

/retest

openshift-ci-robot · 2018-06-28T18:06:53Z

@deads2k: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/verify	`79f9b9b`	link	`/test verify`
ci/openshift-jenkins/integration	`79f9b9b`	link	`/test integration`
ci/openshift-jenkins/gcp	`79f9b9b`	link	`/test gcp`
ci/prow/e2e-gcp	`488d534`	link	`/test e2e-gcp`
ci/prow/conformance-k8s	`98dd779`	link	`/test conformance-k8s`
ci/prow/launch-gcp	`d206b32`	link	`/test launch-gcp`
ci/prow/gcp	`edbddad`	link	`/test gcp`
ci/prow/unit	`edbddad`	link	`/test unit`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

liggitt · 2018-06-28T18:38:12Z

single gcp failure is an existing flake
single unit failure is a new flake from upstream (same units passed in rebase 1.11 force specified kubelet hostname #20129 which differed only by a single commit which doesn't affect unit tests)

merging, will start on followups.

…iags This seems fallout from 1.11 rebase (openshift#20033)

openshift-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 18, 2018

openshift-ci-robot requested review from knobunc and soltysh June 18, 2018 16:49

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2018

deads2k commented Jun 18, 2018

View reviewed changes

deads2k force-pushed the rebase-04-1.11beta2 branch from 3dd4f26 to de18cb9 Compare June 18, 2018 17:47

mfojtik reviewed Jun 19, 2018

View reviewed changes

deads2k force-pushed the rebase-04-1.11beta2 branch from de18cb9 to 180b247 Compare June 19, 2018 21:02

openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 19, 2018

liggitt reviewed Jun 19, 2018

View reviewed changes

openshift-ci-robot requested a review from ingvagabund June 19, 2018 21:48

liggitt reviewed Jun 19, 2018

View reviewed changes

deads2k and others added 4 commits June 28, 2018 07:26

disable volume alpha e2e

7eb1193

disable sysctl tests that don't support SCC. Needs to be reverted

b5133dc

update TSB so that it can handle secrets, but it's still generically …

67bafb1

…broken

deads2k force-pushed the rebase-04-1.11beta2 branch from ed3cecd to d206b32 Compare June 28, 2018 11:28

soltysh reviewed Jun 28, 2018

View reviewed changes

liggitt and others added 3 commits June 28, 2018 07:47

UPSTREAM: 65547: Honor custom transport dialer

0dd985e

Restore TestSNI

57a6ec0

squash to: imagepruning: temporarily disable event handling

7ccf2c8

UPSTREAM: 65587: Revert "certs: only append locally discovered addres…

edbddad

…ses when we got none from the cloudprovider" This reverts commit 7354bbe.

deads2k force-pushed the rebase-04-1.11beta2 branch from 270e67b to edbddad Compare June 28, 2018 16:45

openshift-ci-robot assigned mfojtik Jun 28, 2018

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 28, 2018

liggitt merged commit 8157540 into openshift:master Jun 28, 2018

deads2k deleted the rebase-04-1.11beta2 branch July 3, 2018 17:49

pravisankar pushed a commit to pravisankar/origin that referenced this pull request Aug 15, 2018

Fix provided options object is not a PodLogOptions error in network d…

d4c99ef

…iags This seems fallout from 1.11 rebase (openshift#20033)

pravisankar mentioned this pull request Aug 15, 2018

Bug 1614660 - Network diagnostic will auto detect runtime #20647

Merged

		# pod - sjenning
		# origin-3.11-cadvisor-v0.30.1 MISSING PATCHES

Rebase 1.11 #20033

Rebase 1.11 #20033

Conversation

deads2k commented Jun 18, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liggitt Jun 18, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liggitt Jun 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Jun 18, 2018

Choose a reason for hiding this comment

mfojtik Jun 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfojtik Jun 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mfojtik commented Jun 19, 2018

soltysh commented Jun 19, 2018

deads2k commented Jun 19, 2018

deads2k commented Jun 19, 2018

deads2k commented Jun 19, 2018

deads2k commented Jun 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Jun 28, 2018

deads2k commented Jun 28, 2018

deads2k commented Jun 28, 2018

miminar commented Jun 28, 2018

deads2k commented Jun 28, 2018

deads2k commented Jun 28, 2018

deads2k commented Jun 28, 2018

mfojtik commented Jun 28, 2018

openshift-ci-robot commented Jun 28, 2018

sjenning commented Jun 28, 2018

deads2k commented Jun 28, 2018

openshift-ci-robot commented Jun 28, 2018 • edited Loading

liggitt commented Jun 28, 2018

deads2k commented Jun 18, 2018 •

edited

Loading

liggitt Jun 18, 2018 •

edited

Loading

liggitt Jun 20, 2018 •

edited

Loading

mfojtik Jun 19, 2018 •

edited

Loading

mfojtik Jun 19, 2018 •

edited

Loading

deads2k commented Jun 19, 2018 •

edited

Loading

openshift-ci-robot commented Jun 28, 2018 •

edited

Loading