POSIX special files and NFS reparse points

[billziss-gh]

WSL should support all POSIX special files on DrvFs as NFS reparse points.

Microsoft already has documentation on implementing special files on NFS: https://msdn.microsoft.com/en-us/library/dn617178.aspx

However WSL currently supports only symbolic links on DrvFs. Furthermore they are implemented as a new form of reparse point with tag 0xa000001d and the link target stored in the file's content.

While I understand that NTFS symbolic links cannot adequately store POSIX targets, it is unclear why the existing NFS reparse points could not be used for WSL.

[My apologies for not following your issue template. This was posted accidentally while still being edited.]

[aseering]

Hi @billziss-gh -- thanks for posting! Regarding NFS, though -- did you mean "NTFS"? As far as I know, WSL does not support NFS at all right now. I agree that it would be cool if WSL supported NFS special files, but I would personally like to see it support NFS regular files first :-) Are NFS special files especially important to you / to something that you do?

If you did mean NTFS: WSL does actually support POSIX special devices on NTFS right now. But only in VolFs. VolFs is NTFS too. It's just making extensive (and, at times, fragile) use of NTFS features that are not broadly understood by most ordinary Windows applications.

Do you have a specific need / use for broader support for POSIX special files other than symlinks in DrvFs? There are lots of things out there that the WSL team could implement. They seem to often prioritize the ones that would help the most people first.

Regarding symlinks in DrvFs -- I for one wish they would get rid of the special WSL symlink reparse point and just use regular Windows symlinks. But that's already discussed on #559 .

[billziss-gh]

Actually I did mean NFS as in "Network File System". To clarify: with this issue I am not asking for NFS support; I am only offering Microsoft's own reparse points in NFS as a solution for POSIX special files.

NFS Reparse Points can be implemented on top of any file system that supports reparse points (including of course NTFS). The reasons that I would like to see NFS Reparse Point support are the following:

• NFS Reparse Points already provide a solution for creating POSIX symlinks on Windows file systems. There should not be a need to invent yet another symlink on Windows.
• NFS Reparse Points can already support POSIX special files (LNK, CHR, BLK, FIFO, SOCK).

Regarding symlinks in DrvFs -- I for one wish they would get rid of the special WSL symlink reparse point and just use regular Windows symlinks. But that's already discussed on #559.

This sounds desirable. However the problem is that Windows symlinks are quite different from POSIX ones, so I can see why the WSL folks might have chosen to take an alternative route.

[fpqc]

@billziss-gh Windows also does happen to support mounting NFS shares if you're on Pro or Enterprise, just for the record.

[therealkenc]

@billziss-gh Just out of curiosity, what posix semantics do you see that would be problematic if WSL went with Windows symlinks per #1475?

I did a port of nfs-fuse to Dokan a while back (message), and one of the (several) reasons I abandoned the idea was because Dokan doesn't appear to support symlinks. I didn't know about WinFsp at the time. [One of the other reasons was anyone wanting WSL/Win32 interop is going to expect FindFirstChangeNotification and inotify to work in both directions.]

[billziss-gh]

@fpqc thank you.

I have no direct interest in NFS support for Windows. I think that perhaps the name of these "NFS reparse points" is distracting from my ask.

Reparse points are a general mechanism for attaching special behavior to files on NTFS. Microsoft has used reparse points to add behavior like "mountpoints" (junctions) and "symbolic links" to NTFS and Windows.

Microsoft also had to solve the problem of representing POSIX symbolic links and special files on Windows file systems before. They defined a form of reparse point, which they called "NFS reparse point" (likely because it emerged on NFS). This reparse point supports POSIX symbolic links, but also all common POSIX special files (LNK, CHR, BLK, FIFO, SOCK).

This is the definition of NFS reparse points: https://msdn.microsoft.com/en-us/library/dn617178.aspx

There is a proliferation of symbolic link solutions and it is not a good thing IMO. We have NTFS symlinks (IO_REPARSE_TAG_SYMLINK) and junctions (which are actually a kind of special directory symlink). We have POSIX symlinks through "NFS reparse points" (IO_REPARSE_TAG_NFS). And now we have the WSL symlink (ReparseTag== 0xa000001d ). And I will not even mention the various symlink mechanisms employed by Cygwin.

I would personally like to see one standard for POSIX symlinks on Windows. Although I can see why Microsoft would choose not to reuse NTFS symbolic links (IO_REPARSE_TAG_SYMLINK) for POSIX symlinks, I am not sure why they disregarded the "NFS" ones. Hence my ask.

[There are many issues with NTFS symlinks, but one of the most important ones is that they can be set on both files and directories. In many cases NTFS will refuse to follow a file symlink that points to a directory or vice-versa. IMO this is an even bigger issue than the SeCreateSymbolicLinkPrivilege.]

My apologies if you knew all that and you were just trying to point me to the right direction re: NFS.

[billziss-gh]

@therealkenc wrote:

@billziss-gh Just out of curiosity, what posix semantics do you see that would be problematic if WSL went with Windows symlinks per #1475?

As mentioned in my comment above, the biggest issue (IMO) is that an NTFS symbolic link can be either a directory or a file. The expectation is that symlinks pointing to directories have to be directories themselves and symlinks pointing to files have to be files themselves.

This causes various problems. For example, you cannot create a symlink first and then the file/directory pointed by it (because you do not know whether to create the symlink as directory or file when the target does not exist).

NTFS will in general refuse to follow symlinks that point to the wrong file type , although not consistently. Looking at some of the WinFsp code I see this comment: /* NTFS open with FILE_FLAG_BACKUP_SEMANTICS does not care about SYMLINK/SYMLINKD difference! */.

There is also the issue of SeCreateSymbolicLinkPrivilege, but that to me is a smaller issue. The reason is that Microsoft has many ways of fixing this problem and in fact are now finally doing so.

I did a port of nfs-fuse...

Fantastic! This has been on my interest list for a while. Would you like me to help fully port it to Windows? Drop me a note at the WinFsp project.

[therealkenc]

ran course