Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739

Open
SteveSandersonMS opened this issue Dec 6, 2019 · 164 comments
Labels
feature wsl2 Issue/feature applies to WSL 2

Comments

@SteveSandersonMS
Copy link

WSL2 is really close to being a perfect runtime environment for server apps being developed in Windows. Great job! One missing feature however is breaking a core part of the developer flow.

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.

Notes:

Bug report template

  • Your Windows build number: 10.0.19033.1

  • What you're doing and what's happening:

    This applies to all tools that listen for file change notifications, but as an example take webpack. Repro steps:

    • In your Windows filesystem, create an empty directory (example: c:\repro), and then add these three files to it
    • In a WSL2 Ubuntu 18.04 environment, install Node and NPM: sudo apt-get install nodejs npm
    • Still in WSL2, go into the directory from earlier: cd /mnt/c/repro
    • Restore NPM dependencies: npm i
    • Run Webpack in watch mode: npm run build:watch. Wait a few seconds until it completes the first build. It will now be waiting for further changes to your source files.
    • In a Windows application (e.g., Notepad or Visual Studio), open c:\repro\index.js and save some change to it. For example, change 'Hello, world' to 'Hello, world 2'.
  • What's wrong / what should be happening instead:

    Expected behavior: Webpack should see the change and rebuild. That is, you'll see it log information about another build, and the output in dist/bundle.js will be updated.

    Actual behavior: Webpack doesn't respond at all, because there's no file change notification.

Finally I understand that the fix for this is likely to be "add file watch capabilities to the Plan9 server", and you may feel this is already being tracked by #4064. However #4064 describes a more obscure symptom of this missing feature and makes it sound like an intermittent issue. What I'm reporting here is not intermittent at all, and is a pretty mainstream scenario (using tools like webpack --watch). Thanks!

@therealkenc
Copy link
Collaborator

therealkenc commented Dec 6, 2019

Yes, #4064 is most unfortunately titled. Worse, the OP has no repro steps and an unecessary symlink confusing the direction they are talking about. Craig's notepad.exe repro with cat isn't an inotify(7) (file watcher) problem, even if implementing inotify is the fix to whatever loosly implied sync problems are going on over there.

Nevertheless #4701 got closed as a dupe with (quoth):

We need to add file watch capabilities to the Plan9 server that serves files to a WSL2 distro, and we're tracking that work item here

So, best I can tell, #4064 is being treated as the LZ for inotify triggers from Windows to WSL2. For lack. #4169 is pretty much exactly your use-case also landed dupe #4064.

This did work great on WSL1

Yes, known regress.

@nake89
Copy link

nake89 commented Jan 4, 2020

Is there a temporary fix to this? This is giving me a headache.

Using Windows 10, WSL 2. Running npm run serve on my Vue project, which normally hot reloads when I do changes (on my Linux and Mac, and WSL1 and I think maybe WSL2 before(?)). Now I have to shut down and restart the Vue server.

Will probably just install Linux on this machine. Developing on a Windows machine is a real pain.

@craigloewen-msft
Copy link
Member

@nake89 are you able to move your project over to the Linux root file system? i.e: Store in your Linux home folder for example? Are there any factors that are blocking you from doing so?

And per @therealkenc 's comment this would be a much better landing zone for adding inotify to the 9P file server. I'll update my comment in #4701 to point here, and add some tags to this issue.

@nake89
Copy link

nake89 commented Jan 7, 2020

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

@SteveSandersonMS
Copy link
Author

SteveSandersonMS commented Jan 7, 2020

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS.

@craigloewen-msft
Copy link
Member

@SteveSandersonMS agreed, we will still be tracking this issue here. :)

@ghost
Copy link

ghost commented Jan 23, 2020

Hi, I am encountering this with docker-sync. My setup is:

  • working folder under C:\users for various php projects
  • Edit a file in phpstorm
  • Changes reflected within my WSL2 Ubuntu instance
  • Running docker containers within WSL2 with volumes synced using docker-sync and Unison

If I make a change to a file directly inside Ubuntu, changes are reflected in the container's mounted volume. However, if I edit a file in PhpStorm in Windows, the change is reflected in Ubuntu as expected but not in the container.

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me.

@Carl-Hugo
Copy link

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

@Carl-Hugo
Copy link

Carl-Hugo commented Feb 5, 2020

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

I wrote a small blog post explaining this workaround and talking about a few more things that I discovered/experienced, using Jekyll on WSL2: Speed up your builds to up to 375% and watch for changes for an even faster dev cycle using this workaround on WSL2/Ubuntu, if that can help someone.

Note: that applies to more than just Jekyll, Jekyll it was just the catalyst.

@safizn
Copy link

safizn commented Feb 5, 2020

For those stumbling upon, some notes about WSL2 features & limitations (OS build 19041.21, insiders slow ring):

  • inotify filesystem events are not propagated between WSL2 & Windows. Although it will be supported in future releases as stated in Microsoft documentation.
  • Accessing Windows filesystem from WSL2, when developing is extremely slow. While moving projects to WSL2 filesystem, will increase performance, much faster than WSL1 & Windows development. (WSL2 can be accessed in path \\wsl$\)
  • VSCode installed in Windows, with remote extension pack, will install VSCode server automatically in WSL2. Some extensions should be installed in the WSL2 side to work, when openning files from the local WSL2 filesystem. The VSCode Extenions tab in the UI provides indications and guides through the required changes.
  • localhost is managed for you by Windows, and allows access to WSL2. Some cases require accessing the WSL2 VM by it's IP directly.
  • Symlinks in WSL2 work seamlessly between WSL2, Docker containers, & Windows, which wasn't the case with WSL1. Using Docker Desktop on WSL2 experimental feature.
  • Some graphical programs (e.g. SmartGit/GitKraken) need to be installed in WSL2 and accessed through GUI client through Windows (Unix X server), to overcome the inotify events & performance degredations.

@ivellios
Copy link

ivellios commented Mar 9, 2020

While I am having similar issue with my React App and I am looking for a solution to this as well, I can reply to:

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me.
@weknowsoftware

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS.
@SteveSandersonMS

This hint by @myuseringithub:

  • Accessing Windows filesystem from WSL2, when developing is extremely slow. While moving projects to WSL2 filesystem, will increase performance, much faster than WSL1 & Windows development. (WSL2 can be accessed in path \\wsl$\)

is a great one to help you! Also I found out yesterday that since \\wsl$\ is a network resource in Windows you can easily map it to a drive. Then in ANY editor/tool you can use it as if it was on your computer. Just go to the Explorer and manually enter this address to access the resource. You will most likely see "Ubuntu" folder. It is a root of your WSL and it can be mapped to the drive (right click -> map to drive). That helped me setup my project on WSL natively, while being able to edit in the windows editor.

Though now I am struggling with Docker containers permissions, which I hope to solve separately and see if reloads in my app work.

Hope that helps.

@nicks
Copy link

nicks commented Mar 27, 2020

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

@Carl-Hugo
Copy link

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

You can use the Linux file system directly, accessible from Windows too; see #4739 (comment)

@mattlacey
Copy link

@Carl-Hugo Does that work for you, as in, are changed detected? I've been using the \wsl$ path with windows editors for a while because with WSL2 the Linux FS is so much faster, but even when using windows editors with files stored there it's not triggering HMR when using npm watch.

@Carl-Hugo
Copy link

@mattlacey Yes it works...

  1. image
  2. ctrl+s the README.md file in VS Code (Windows)
  3. image

Side note: for-the-new-order is not a sect but a Star Wars RPG thing 😉

@vielhuber
Copy link

Some graphical programs (e.g. SmartGit/GitKraken) need to be installed in WSL2 and accessed through GUI client through Windows (Unix X server), to overcome the inotify events & performance degredations.

Can you please elaborate on that: Is this working well? Any tutorials / starting points how to set this up?

@safizn
Copy link

safizn commented Apr 17, 2020

@vielhuber I wrote it as a comment for myself, tried to implement it once without success. Just wait till WSL2 will support inotify. You could still use graphical interfaces for git, only that you would have to constantly refresh to see changes, which is the same case when dealing with VSCode's git panel.

But if you wish to dig deeper, check out:

I wrote these comments when setting up my development environment

@vielhuber
Copy link

@myuseringithub Awesome thank you.

@huysentruitw
Copy link

I use my regular Git client in Windows, because my project is still on my Windows machine. However, to get livereload working, I simply add a symlink from my WSL2 home directory to the project on my windows machine, f.e.:

ln -s /mnt/d/projects/contrib/create-your-future-website/ ~/create-your-future-website

After that, I start jekyll serve --livereload from ~/create-your-future-website and open that folder in VS code using ctrl-shift-p and then type "Remote-WSL: New Window" as explained above.

This way, you don't have to move your project and can still enjoy your favorite git client in Windows. Profit!

@huysentruitw
Copy link

Arg, scratch the above idea. It only seems to get triggered by file changes at the top-level directory. 😢

karisal-anders added a commit to City-of-Helsinki/palvelutarjotin-ui that referenced this issue Oct 21, 2024
now "docker compose up --build" spins up a development build locally,
hot reloading on Windows 11 + Docker Desktop didn't work, but that
may be Windows + WSL2 specific setup problem, see the following:
- vercel/next.js#36774
- microsoft/WSL#4739

also:
 - remove deprecated version number from docker-compose.yml

refs PT-1792
karisal-anders added a commit to City-of-Helsinki/palvelutarjotin-ui that referenced this issue Oct 22, 2024
now "docker compose up --build" spins up a development build locally,
hot reloading on Windows 11 + Docker Desktop didn't work, but that
may be Windows + WSL2 specific setup problem, see the following:
- vercel/next.js#36774
- microsoft/WSL#4739

also:
 - remove deprecated version number from docker-compose.yml

refs PT-1792
@shyce

This comment has been minimized.

@sdserage

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@shyce

This comment has been minimized.

@shyce

This comment has been minimized.

@seanballais

This comment has been minimized.

@floyd-may

This comment has been minimized.

@sdserage

This comment has been minimized.

@microsoft microsoft locked as too heated and limited conversation to collaborators Oct 25, 2024
@craigloewen-msft
Copy link
Member

Hi folks, temporarily locking this thread for now. As a reminder this project follows the Microsoft code of conduct. We're still tracking this as a known issue, and if you're experiencing this you can see some of the alternative solutions in the thread above to help it, such as using VS Code Remote, using \\wsl.localhost\ to access the files from the Linux file system, etc.

Thank you!

@elad-levy-bl
Copy link

legacy dotnet code in my project (only runs on windows) prevents me from using the suggested workaround. Is there a plan to support this anytime soon?

@seanballais
Copy link

@eladlevy-brainlab, just curious. Which workarounds did you try already?

@elad-levy-bl
Copy link

I tried several forms of tail -f but nothing worked, and the suggested workaround was to move the project files to wsl which as I said is problematic for my use case.
Actually, since posting this comment, I did find that using the powershell equivalent Get-Content $log_file -Wait does work and keeps track of changes in the file, even when running on wsl for files on the windows filesystem - so that's another workaround.

@floyd-may
Copy link

Get-Content -Wait just uses a blocking read and doesn't rely on filesystem events.

Any chance we could get a Microsoft person to give us a current status update? It's hard to tell with this being such a long thread. Might also be worth editing the topmost post (if that's possible?) so that folks can easily see the current status.

@seanballais
Copy link

Just pinging in, @craigloewen-msft, to bring this one up his way closer. He might have an update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature wsl2 Issue/feature applies to WSL 2
Projects
None yet
Development

No branches or pull requests