cache add: transferring cached image: Process exited with status 137 from signal KILL

[perrywang]

problem
I am adding offline ~2G image to minikube with cache add command but it always failed.
When I ssh login VM I found a ~400M image file in /tmp folder. I try a lot of times it seems it is always failed at ~400M data transfered.
Small size image has no such problems.
The strange is when I put this image file in files folder it is always transferred to VM successfully. And also I can run docker load -i xxx successfully in VM.
logs
! Failed to cache and load images: loading cached images: loading image {MINI_HOME}\cache\images\xxxx-web_2.89.27:
transferring cached image: Process exited with status 137 from signal KILL
environment:
windows 10 + virtualbox
minikube 1.0.0

[afbjorklund]

Seems to indicate an issue with transferring the image, like a scp timeout ?

[perrywang]

Seems to indicate an issue with transferring the image, like a scp timeout ?

I am also doubt timeout.
The strange is put image in files folder it can be transferred to VM successfully. Confusing.

[afbjorklund]

If you scp the image yourself, does it transfer OK to the VM ? 2G is slighly bigger than what CI tests :-)

[perrywang]

If you scp the image yourself, does it transfer OK to the VM ? 2G is slighly bigger than what CI tests :-)

I doubt it is relating with default target /tmp it is different partition only with ~2G size.
Could team consider expose this arg to user?

[afbjorklund]

Ah, right. What you are saying is that "minikube cache add" copies to /tmp which is on tmpfs (RAM)!
So that 400M limit is probably how much RAM you have available (free -m), after starting kubernetes.

We could definitely use something on the data disk instead, 20G instead of 2G... Already do, for DVP:

/dev/sda1 on /tmp/hostpath_pv type ext4 (rw,relatime,data=ordered)
/dev/sda1 on /tmp/hostpath-provisioner type ext4 (rw,relatime,data=ordered)

Since we delete the images after loading them into the container runtime, it normally doesn't matter.

[perrywang]

Ah, right. What you are saying is that "minikube cache add" copies to /tmp which is on tmpfs (RAM)!
So that 400M limit is probably how much RAM you have available (free -m), after starting kubernetes.

We could definitely use something on the data disk instead, 20G instead of 2G... Already do, for DVP:

/dev/sda1 on /tmp/hostpath_pv type ext4 (rw,relatime,data=ordered)
/dev/sda1 on /tmp/hostpath-provisioner type ext4 (rw,relatime,data=ordered)

Since we delete the images after loading them into the container runtime, it normally doesn't matter.

Yes. @afbjorklund findmnt show such mounts. The problem is even I have more memory df -h still show /tmp only 2G available. It seems /tmp mounted with such options. Currently as workaround I use files directory to transfer to /data which was mounted from /dev/sda1/data.