Corrupt synced folder cache using Vagrant 1.4.2 and VMWare provider

[danielbachhuber]

Using Vagrant 1.4.2, the VMWare provider (VMWare Version 6.0.2-1398658) and Ubuntu 13.04, I'm regularly experiencing corrupted synced folder caches when editing synced files using Sublime Text on my Mac.

Given this edit history on a sample file, the file consistently ends up as the following inside the VM (notice the incorrect file termination point):

Making another edit to the file (even whitespace) will fix the cached representation of the file. The issue can also be resolved by running echo 1 > /proc/sys/vm/drop_caches as sudo inside the VM.

My co-worker @joehoyle runs a similar setup with Vagrant 1.3.3 and hasn't ever experienced the issue. I haven't yet tested downgrading because I upgraded the VMWare provider which requires 1.4.0

Any suggestions as to what the cause might be, or what additional debugging I can do? I switched to VMWare because of caching issues associated with VirtualBox and NFS, and it's frustrating to say the least that one caching issue has been replaced by another.

[mitchellh]

This is a bug with VMware 6.0.2 guest additions. It is sporadic. Downgrade to fix.

[danielbachhuber]

Thanks for the reply, @mitchellh. Downgrading to 6.0.1 didn't fix the problem. I've now downgraded to 5.0.4 and am seeing this issue:

[default] Starting the VMware VM...
An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:

Command: ["start", "/Users/danielbachhuber/salty-wordpress/.vagrant/machines/default/vmware_fusion/5f9afd2d-5363-4869-95c2-6889cc7eb141/raring64.vmx", "nogui", {:notify=>[:stdout, :stderr]}]

Stdout: 2014-01-06T10:28:56.534| ServiceImpl_Opener: PID 916
Error: Unknown error

Stderr:

Destroying the box and re-provisioning doesn't resolve.

[mitchellh]

@danielbachhuber The bug is in the guest additions, not the host VMware.

[danielbachhuber]

Derp derp. Here are some related threads I came across:

• https://communities.vmware.com/message/2314983#2314983
• https://communities.vmware.com/message/2326272

Based on those threads, I then performed the following steps inside my VM:

1. Checked my current version of VMWare Tools: vmware-config-tools.pl -h

VMware Tools 9.2.2 build-893683 configurator.

1. Downloaded VMWare Tools for 6.0.1: cd /tmp; wget https://softwareupdate.vmware.com/cds/vmw-desktop/fusion/6.0.1/1331545/packages/com.vmware.fusion.tools.linux.zip.tar
2. Extracted and mounted the ISO: tar -xvf com.vmware.fusion.tools.linux.zip.tar; unzip com.vmware.fusion.tools.linux.zip; mkdir /mnt/vmware-tools; mount -o loop linux.iso /mnt/vmware-tools/
3. Ran the upgrader (I have a 64-bit box): cd /mnt/vmware-tools; ./vmware-tools-upgrader-64

However, after doing a vagrant reload, I seem to be back at the same version and build number of VMWare Tools. I haven't been able to reproduce the issue though, so maybe it's been fixed. I'll give it a few days before I make the determination.

[danielbachhuber]

Hah. Now when I try to cd into a synced folder inside the VM my user session crashes. Fantastic.

[danielbachhuber]

Here's the call trace for the kernel panic (from VMWare GUI):

[danielbachhuber]

I did vagrant destroy to get back to square one. @joehoyle is on VMware Tools 9.6.0 build-1294478 with no issues, so I guess I'll attempt to find a copy of that version or earlier tomorrow and install it.

@bdcravens I see a similar report from you in #2533. Did you manage to work out a solution?

[danielbachhuber]

Actually, I just switched to NFS after installing nfs-common inside the VM and it seems to work alright. We originally switched away from NFS because it had caching issues with VirtualBox. I did some quick testing and that particular caching issue doesn't seem to be manifesting itself.

[bdcravens]

@danielbachhuber No the underlying issue wasn't resolved, but like you, I switched to NFS and it seemed to resolve the behavior.

[andyfowler]

Just throwing this here in case it helps: my team was all having issues with NFS not updating files if the file length stayed the same. In our case, we were all OS X host, Ubuntu guest, Virtualbox provider, NFS synced folder. The fix was to stop using atomic saves in Sublime Text (which is the default, at least in ST3). In ST: Sublime Text > Preferences > Settings- User

{
    "atomic_save": false
}

[joehoyle]

@andyfowler it's possible you just solved a very long running problem... will report back!

[andyfowler]

Please do! This one was killing us for months now. Trying to get the word out, because I know I've seen quite a few others with it... https://twitter.com/andyfowler/status/420605123568472065

[danielbachhuber]

Just throwing this here in case it helps: my team was all having issues with NFS not updating files if the file length stayed the same

I think this is just an issue with VirtualBox, though. I did some basic testing yesterday with VMWare and didn't run into this problem (but had it all the time with VB). I'll report back if I run into it again with VMWare.

[andyfowler]

@danielbachhuber I've been doubtful that this is a VirtualBox issue, since it's really just the NFS connection between host and guest, which is all via network, and not any VirtualBox magic.

But to your original, our issue was when editing files that didn't change in size: I haven't experienced any actual corruption via NFS. I had a hell of a time using the VirtualBox native shared folders, though, which is what pushed us to NFS.

[astockwell]

Indeed @andyfowler you are correct, not just a VirtualBox issue! Was having the same issue with an Ubuntu guest (12.04 LTS) on VMWare (latest), OS X host (10.9.1), running ST3 (latest) and also a ruby (2.1.0) Guard post-processor. Narrowed it down to the VMWare guest literally not seeing updates to files within the shared folder when the file size didn't change. NFS seems to have solved it! Many thanks as this would have been a deal-breaker for MAMP-killing.

[ladyrassilon]

I'm still seeing an issue with this, I'm just using conventional (not NFS) syncing, and the only way I can get the syncing to work is to do this every time I make a change.

sudo echo 1 > /proc/sys/vm/drop_caches

Every time I want to sync.

I don't get any problems with virtualbox, I'm using 1.5.1, but this has been a problem since 1.4.2 (I'm just going to test again with 1.5.2 to see if that works)

Its the reason I'm forced to use virtualbox as opposed to vmware.

[glucas]

Running Vagrant 1.5.3 on Windows, VMware Workstation, and an Ubuntu 12.04 guest. I'm (intermittently?) seeing synced folders get out of date. When this happens I can see the files up to date under /mnt/hgfs/-xxx, but the path where vagrant has mounted the folder is out of sync.

@ladyrassilon - thanks, at least this gives me a way to force the sync.

[nacht]

this ticket appears to be closed - but want to add to the fact that our organisation is seeing this. Currently running
Vagrant 1.5.2
OS X 10.9.2
VMWare Fusion Pro 6.0.2
with Ubuntu 12.04 guest

[helderco]

@nacht try to upgrade your vmware to 6.0.3

[adambiggs]

@helderco VMWare 6.0.3 did nothing to solve the problem for me...

[ladyrassilon]

Since Vmware (fusion) 6.0.3 I've had no folder syncing issues.

Did you start from a fresh vm?

[adambiggs]

I tried vagrant destroy/vagrant up.

I'm not 100% sure it's the same issue, but using VMWare's shared folder, I get all kinds of strange errors when trying to run composer for a Symfony2 project. But when switching to NFS, the problems go a way (leaving me with a different set of NFS problems...).

[glenbot]

I still had this issue in VMware Fusion 6.0.3 and I wrote up a post on how I fixed it. http://codrspace.com/glenbot/fix-vagrant-vmhgfs-file-corruption-bug-in-vmware-6-0-2-and-6-0-3/

[atombender]

This issue should be reopened. Experiencing this problem with Virtualbox (latest, 4.3.22 r98236, image is Ubuntu 12.04 with kernel 3.2.0-23-generic) and ordinary vboxsf mounts. @ladyrassilon's manual VM cache drop workaround resolves the problem every time.

[rvdh]

I agree with @atombender - I'm experiencing this issue as well. Debian 8 image with latest VirtualBox Version 5.0.14 r105127.