Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Defer dashboard deployment until "minikube dashboard" is executed #3485

Merged
merged 6 commits into from
Jan 11, 2019
Merged

Defer dashboard deployment until "minikube dashboard" is executed #3485

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9ca9242
Disable dashboard by default
tstromberg Dec 21, 2018
dba0c0a
Disable dashboard by default
tstromberg Dec 21, 2018
f3d1581
Add helpful dashboard status messages to stderr
tstromberg Dec 21, 2018
44a230b
Remove dashboard deployment check from TestPersistence - it's no long…
tstromberg Dec 21, 2018
6f5c380
RunDaemon: tee stderr for debugging
tstromberg Dec 21, 2018
1bd4b05
Increase dashboard retry time from 30 seconds to 3 minutes
tstromberg Dec 21, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions cmd/minikube/cmd/dashboard.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ import (
"github.com/pkg/browser"
"github.com/pkg/errors"
"github.com/spf13/cobra"
configcmd "k8s.io/minikube/cmd/minikube/cmd/config"
"k8s.io/minikube/pkg/minikube/cluster"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/machine"
"k8s.io/minikube/pkg/minikube/service"

"k8s.io/minikube/pkg/util"
)

Expand Down Expand Up @@ -65,19 +65,30 @@ var dashboardCmd = &cobra.Command{
}
cluster.EnsureMinikubeRunningOrExit(api, 1)

fmt.Fprintln(os.Stderr, "Enabling dashboard ...")
// Enable the dashboard add-on
err = configcmd.Set("dashboard", "true")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this idempontent? If yes, then we don't need "Enabling dashboard ..." the second time (I think addons enable is idempotent) - would be nice to check for enablement.

if err != nil {
fmt.Fprintf(os.Stderr, "Unable to enable dashboard: %v\n", err)
os.Exit(1)
}

ns := "kube-system"
svc := "kubernetes-dashboard"
if err = util.RetryAfter(30, func() error { return service.CheckService(ns, svc) }, 1*time.Second); err != nil {
fmt.Fprintln(os.Stderr, "Verifying dashboard health ...")
if err = util.RetryAfter(180, func() error { return service.CheckService(ns, svc) }, 1*time.Second); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be interesting to keep a "watch" on the dashboard pod + on a separate line the result of the service status? But this idea might be better off in a separate PR. It's just 3 minutes of potential waiting is definitely long. Out of those 1 minute is almost guaranteed to be just the addon-manager kicking in...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good idea, but due to the complexity l will leave this for another PR. This step generally only takes a second or two, but to avoid flakes in places with poor connectivity, I wanted to make sure to wait an extended period for the pod to come up.

fmt.Fprintf(os.Stderr, "%s:%s is not running: %v\n", ns, svc, err)
os.Exit(1)
}

fmt.Fprintln(os.Stderr, "Launching proxy ...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fmt.Fprintln(os.Stderr, "Launching proxy ...")
fmt.Fprintln(os.Stderr, "Launching dashboard proxy ...")

p, hostPort, err := kubectlProxy()
if err != nil {
glog.Fatalf("kubectl proxy: %v", err)
}
url := dashboardURL(hostPort, ns, svc)

fmt.Fprintln(os.Stderr, "Verifying proxy health ...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fmt.Fprintln(os.Stderr, "Verifying proxy health ...")
fmt.Fprintln(os.Stderr, "Verifying dashboard proxy health ...")

if err = util.RetryAfter(60, func() error { return checkURL(url) }, 1*time.Second); err != nil {
fmt.Fprintf(os.Stderr, "%s is not responding properly: %v\n", url, err)
os.Exit(1)
Expand Down
2 changes: 1 addition & 1 deletion pkg/minikube/assets/addons.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ var Addons = map[string]*Addon{
constants.AddonsPath,
"dashboard-svc.yaml",
"0640"),
}, true, "dashboard"),
}, false, "dashboard"),
"default-storageclass": NewAddon([]*BinDataAsset{
NewBinDataAsset(
"deploy/addons/storageclass/storageclass.yaml",
Expand Down
4 changes: 0 additions & 4 deletions test/integration/persistence_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,6 @@ func TestPersistence(t *testing.T) {
}

verify := func(t *testing.T) {
if err := util.WaitForDashboardRunning(t); err != nil {
t.Fatalf("waiting for dashboard to be up: %v", err)
}

if err := util.WaitForBusyboxRunning(t, "default"); err != nil {
t.Fatalf("waiting for busybox to be up: %v", err)
}
Expand Down
25 changes: 12 additions & 13 deletions test/integration/util/util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ func (m *MinikubeRunner) Run(cmd string) error {
func (m *MinikubeRunner) Copy(f assets.CopyableFile) error {
path, _ := filepath.Abs(m.BinaryPath)
cmd := exec.Command("/bin/bash", "-c", path, "ssh", "--", fmt.Sprintf("cat >> %s", filepath.Join(f.GetTargetDir(), f.GetTargetName())))
Logf("Running: %s", cmd)
Logf("Running: %s", cmd.Args)
return cmd.Run()
}

Expand Down Expand Up @@ -144,6 +144,17 @@ func (m *MinikubeRunner) RunDaemon(command string) (*exec.Cmd, *bufio.Reader) {
if err != nil {
m.T.Fatalf("stdout pipe failed: %s %v", command, err)
}
stderrPipe, err := cmd.StderrPipe()
if err != nil {
m.T.Fatalf("stderr pipe failed: %s %v", command, err)
}

var errB bytes.Buffer
go func() {
if err := commonutil.TeePrefix(commonutil.ErrPrefix, stderrPipe, &errB, Logf); err != nil {
m.T.Logf("tee: %v", err)
}
}()

err = cmd.Start()
if err != nil {
Expand Down Expand Up @@ -295,18 +306,6 @@ func WaitForBusyboxRunning(t *testing.T, namespace string) error {
return commonutil.WaitForPodsWithLabelRunning(client, namespace, selector)
}

func WaitForDashboardRunning(t *testing.T) error {
client, err := commonutil.GetClient()
if err != nil {
return errors.Wrap(err, "getting kubernetes client")
}
if err := commonutil.WaitForDeploymentToStabilize(client, "kube-system", "kubernetes-dashboard", time.Minute*10); err != nil {
return errors.Wrap(err, "waiting for dashboard deployment to stabilize")
}

return nil
}

func WaitForIngressControllerRunning(t *testing.T) error {
client, err := commonutil.GetClient()
if err != nil {
Expand Down