Mapping of create_options and granted_access

[robert-boulanger]

This should be labelled as question.
When dealing with winfsp in fuse mode, open in operations will be called as
def open(self, path, flags):
in winfspy open looks like
def open(self, file_name, create_options, granted_access):

so far so good.

Now let's assume I have (a kind of) passthrough filesystem.
using fuse i can open the real file which is somewhere in the ntfs filesystem this way:

FSP_FUSE_UF_HIDDEN = 0x00008000
    FSP_FUSE_UF_READONLY = 0x00001000
    FSP_FUSE_UF_SYSTEM = 0x00000080
    FSP_FUSE_UF_ARCHIVE = 0x00000800

    CREATE_NEW = 1
    CREATE_ALWAYS = 2
    OPEN_EXISTING = 3
    OPEN_ALWAYS = 4
    TRUNCATE_EXISTING = 5
    FILE_SHARE_READ = 0x00000001
    FILE_SHARE_WRITE = 0x00000002
    FILE_SHARE_DELETE = 0x00000004
    FILE_SHARE_VALID_FLAGS = 0x00000007
    FILE_ATTRIBUTE_READONLY = 0x00000001
    FILE_ATTRIBUTE_NORMAL = 0x00000080
    FILE_ATTRIBUTE_TEMPORARY = 0x00000100
    FILE_FLAG_DELETE_ON_CLOSE = 0x04000000
    FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000
    FILE_FLAG_RANDOM_ACCESS = 0x10000000
    GENERIC_READ = 0x80000000
    GENERIC_WRITE = 0x40000000
    DELETE = 0x00010000
    NULL = 0

    _ACCESS_MASK = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
    _ACCESS_MAP = {os.O_RDONLY: GENERIC_READ,
                   os.O_WRONLY: GENERIC_WRITE,
                   os.O_RDWR: GENERIC_READ | GENERIC_WRITE}

    _CREATE_MASK = os.O_CREAT | os.O_EXCL | os.O_TRUNC
    _CREATE_MAP = {0: OPEN_EXISTING,
                   os.O_EXCL: OPEN_EXISTING,
                   os.O_CREAT: OPEN_ALWAYS,
                   os.O_CREAT | os.O_EXCL: CREATE_NEW,
                   os.O_CREAT | os.O_TRUNC | os.O_EXCL: CREATE_NEW,
                   os.O_TRUNC: TRUNCATE_EXISTING,
                   os.O_TRUNC | os.O_EXCL: TRUNCATE_EXISTING,
                   os.O_CREAT | os.O_TRUNC: CREATE_ALWAYS}


    def os_open(file, flags, mode=0o777, *, share_flags=FILE_SHARE_VALID_FLAGS):
        '''
        works better then os.open() in windows
        '''
        if not isinstance(flags, int) and mode >= 0:
            raise ValueError('bad flags: %r' % flags)

        if not isinstance(mode, int) and mode >= 0:
            raise ValueError('bad mode: %r' % mode)

        if share_flags & ~FILE_SHARE_VALID_FLAGS:
            raise ValueError('bad share_flags: %r' % share_flags)

        access_flags = _ACCESS_MAP[flags & _ACCESS_MASK]
        create_flags = _CREATE_MAP[flags & _CREATE_MASK]
        attrib_flags = FILE_ATTRIBUTE_NORMAL

        if flags & os.O_CREAT and mode & ~0o444 == 0:
            attrib_flags = FILE_ATTRIBUTE_READONLY

        if flags & os.O_TEMPORARY:
            share_flags |= FILE_SHARE_DELETE
            attrib_flags |= FILE_FLAG_DELETE_ON_CLOSE
            access_flags |= DELETE

        if flags & os.O_SHORT_LIVED:
            attrib_flags |= FILE_ATTRIBUTE_TEMPORARY

        if flags & os.O_SEQUENTIAL:
            attrib_flags |= FILE_FLAG_SEQUENTIAL_SCAN

        if flags & os.O_RANDOM:
            attrib_flags |= FILE_FLAG_RANDOM_ACCESS

        h = _winapi.CreateFile(file, access_flags, share_flags, NULL,
                               create_flags, attrib_flags, NULL)

        return msvcrt.open_osfhandle(h, flags | os.O_NOINHERIT)

Means I get the access_flags and create_flags out of flags passed by the fuse operation open.

When create_options & CREATE_FILE_CREATE_OPTIONS.FILE_NON_DIRECTORY_FILE are not 0, I know to create a file by using
0x40000000 for access_flags and 0x4 for create_flags.

But I can''t find any valid mapping from CREATE_FILE_CREATE_OPTIONS to find the proper parameters for the winapi CreateFile method.

How can this be adressed ?
Or even asked the other way around, how can the create_options and granted_access flags from winfspy open operations be converted to flags which can be used with standard os.open() ?

[vxgmichel]

I'm not sure how to do that, but maybe take inspiration from the winfsp passthrough example?

https://github.com/billziss-gh/winfsp/blob/aedf01a384571219442d98753ba135593cfa1a14/tst/passthrough-cpp/passthrough-cpp.cpp#L412-L445

[robert-boulanger]

I did that already, but it seems not to be 100% complete or correct.
While testing here I find one special issue, which is really tricky:

• Copy an old Word.doc (format 2003 or so) to the filesystem.
• Open it with Word from Office 365 (current version)
• Edit it.
• Try to save

It failes at the last write operation (constrained write) with the error message bad file descriptor.

Here the code to for open and creation which i seperated for better debugging.

def os_create(file, create_options, granted_access,attrib_flags= FILE_ATTRIBUTE_NORMAL):
    CreateFlags = FILE_FLAG_BACKUP_SEMANTICS
    if create_options & CREATE_FILE_CREATE_OPTIONS.FILE_DELETE_ON_CLOSE:
        CreateFlags |= FILE_FLAG_DELETE_ON_CLOSE
    h= ctypes.windll.kernel32.CreateFileW(
        file,
        granted_access,
        FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
        NULL,
        CREATE_NEW,
        CreateFlags | attrib_flags ,
        NULL)

    return msvcrt.open_osfhandle(h, os.O_BINARY | os.O_NOINHERIT )

def os_open(file, create_options, granted_access, attrib_flags= FILE_ATTRIBUTE_NORMAL, *, share_flags=FILE_SHARE_VALID_FLAGS):
    CreateFlags = FILE_FLAG_BACKUP_SEMANTICS
    if create_options & CREATE_FILE_CREATE_OPTIONS.FILE_DELETE_ON_CLOSE:
        CreateFlags |= FILE_FLAG_DELETE_ON_CLOSE

    h= ctypes.windll.kernel32.CreateFileW(
        file,
        granted_access,
        FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
        NULL,
        OPEN_EXISTING,
        CreateFlags,
        NULL)
    return msvcrt.open_osfhandle(h, os.O_BINARY )

When passing os.O_NOINHERIT also to the msvcrt in the open method, most office write operations will fail, the solution above, which is taken 1:1 from bills example works in all cases except this "saving old word files" problem. so I assume, something is missing here, but I can't find out what it is exactly.
So I assume just checking for CREATE_FILE_CREATE_OPTIONS.FILE_DELETE_ON_CLOSE is not enough.

[robert-boulanger]

Problem adressed but still not solved.
When saving old file formats, word is messsing around with the securitydescriptor a lot.
Also Acrobat behaves in this manner when saving PDF files.
Within memfs the security descriptor is just stored in memory and returned when requested. Here everything is fine.
But when the filesystem is persistent, there is no way to store the security descriptor, since in lib there is a method called
ConvertStringSecurityDescriptorToSecurityDescriptorW
but
ConvertSecurityDescriptorToStringSecurityDescriptorW
is missing.
So in this case it is not possible to return the security descriptor which was set by any application two or three open/close cycles before.
Using win32security.ConvertSecurityDescriptorToStringSecurityDescriptor does not help out since the winfsp securitydescriptor in not a PYSecurityDescriptor.
Since winfspy' s Security descriptor is just a named tuple containing the handle and the size, this also won't fit for the win32security methods, but is there any way to convert it to a PYSecuritydescriptor ?

[vxgmichel]

But when the filesystem is persistent, there is no way to store the security descriptor, since in lib there is a method called
ConvertStringSecurityDescriptorToSecurityDescriptorW but ConvertSecurityDescriptorToStringSecurityDescriptorW is missing.

I see, so all you need is a way to serialize security descriptors in order to store them persistently right? In this case, exposing ConvertSecurityDescriptorToStringSecurityDescriptorW through a SecurityDescriptor.to_string() method should be enough right?

This has been already done for ConvertStringSecurityDescriptorToSecurityDescriptorW so it shouldn't be too hard:

winfspy/src/_cffi_src/winfsp.cdef.h

Lines 174 to 179 in 5c34d02

	BOOL ConvertStringSecurityDescriptorToSecurityDescriptorW(
	LPCWSTR StringSecurityDescriptor,
	DWORD StringSDRevision,
	PSECURITY_DESCRIPTOR *SecurityDescriptor,
	PULONG SecurityDescriptorSize
	);

I can make a PR for that once you confirm that this is something useful to your usecase :)

[robert-boulanger]

Confirmed ;)
Yes, this is the missing link, otherwise you never will get persistent filesystems with security.
Also tests will always fail, 10_GetSetSecurity.t can never work currently with a persistent fs.

[vxgmichel]

SecurityDecscriptor.to_string() has been introduced in version 0.8.1

Thanks for the report :)

[robert-boulanger]

0.8.1 solved the problem.
SecurityDescriptor.to_string() works like a charm.
Thanks a lot 💯