Enable support for offline kic

[priyawadhwa]

This PR does a couple things:

1. For --download-only=true, if the docker daemon isn't running, store the kic base image as a tarball on the host machine
2. Before trying to pull the kic base image, check if it exists as a tarball in the cache and load it into the daemon if it does. If we can load the image into the daemon successfully, strip the base image if its digest, since docker doesn't include the digest when loading images into the daemon.

This way, we can download all required artifacts to the host machine, and later run the docker driver offline (I tested this locally and it works).

This should fix one of the bugs around running minikube in Cloud Shell.

Fixes #8366

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [priyawadhwa]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[priyawadhwa]

/ok-to-test

[minikube-pr-bot]

kvm2 Driver
docker Driver

[codecov-commenter]

Codecov Report

Merging #8417 into master will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #8417      +/-   ##
==========================================
- Coverage   34.13%   34.07%   -0.06%     
==========================================
  Files         153      153              
  Lines        9808     9825      +17     
==========================================
  Hits         3348     3348              
- Misses       6058     6075      +17     
  Partials      402      402              

Impacted Files	Coverage Δ
cmd/minikube/cmd/start.go	13.47% <0.00%> (-0.05%)	⬇️
pkg/minikube/image/image.go	6.00% <0.00%> (-1.06%)	⬇️

[medyagh]

thanks for this PR, this PR is doing the right thing and improves kic experience.
do you mind manually making sure, the chosen Fallback image, survives Stopping cluster and starting it again. just to verify the chosen fallback will be presistiantly used across stop/start

[medyagh]

/ok-to-test

[medyagh]

minor nit

[priyawadhwa]

confirmed that this works locally

[minikube-pr-bot]

kvm2 Driver
Times for minikube: [63.98777553 66.44593302400001 64.345435317]
Average time for minikube: 64.92638129033332

Times for Minikube (PR 8417): [63.926408698 64.834971762 62.21764555799999]
Average time for Minikube (PR 8417): 63.65967533933334

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8417) |
+--------------------------------+-----------+--------------------+
| * minikube v1.11.0 on Debian   |  0.056622 |           0.058520 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.022692 |           0.023854 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003133 |           0.003264 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.715412 |          40.428956 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.3 | 22.260525 |          21.102802 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.276689 |           1.380420 |
| components...                  |           |                    |
| * Enabled addons:              |  0.503068 |           0.586023 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.084453 |           0.071623 |
| configured to use "minikube"   |           |                    |
|                                |  0.003785 |           0.004214 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.494778824999997 26.195681180999998 26.043728661]
Average time for minikube: 26.244729555666666

Times for Minikube (PR 8417): [25.420945449999994 26.133872598 26.057333433]
Average time for Minikube (PR 8417): 25.870717160333328

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 8417) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.11.0 on Debian           |  0.072541 |           0.076196 |
|                                   9.11 |           |                    |
| * Using the docker driver              |  0.002738 |           0.002877 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.060426 |           0.059867 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.653471 |           7.534828 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.3         |  0.118306 |           0.116916 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.032097 |          16.897807 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.180541 |           1.108839 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.054539 |           0.002664 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.066807 |           0.065770 |
| configured to use "minikube"           |           |                    |
|                                        |  0.003264 |           0.004953 |
+----------------------------------------+-----------+--------------------+

[medyagh]

Docker Driver : TestFunctional/parallel/DockerEnv is related to this flake:
#8427

https://storage.googleapis.com/minikube-builds/logs/8417/1109616/Docker_Linux.html#fail_TestFunctional%2fparallel%2fDockerEnv