/utilities/stat mode return value has been broken

[chrisjsewell]

In 8fa4044, @aledabin changed the code I added (#133) from returning %f to %a for the mode, this has broken its use 😅

Let me explain, for a simple (empty) file:

$ stat -c '%a' example.txt
644
$ stat -c '%f' example.txt
81a4

Here 81a4 is the hexidecimal encoding of BOTH the permissions and the file type.
For example, using Python you can do:

import stat
# recover the permissions as octahedral
oct(int("81a4", 16) & 0o777)[2:] == "644"
# recover the permissions as string
stat.filemode(int("81a4", 16)) == "-rw-r--r--"
# check the file type
stat.S_ISREG(int("81a4", 16)) is True
stat.S_ISDIR(int("81a4", 16)) is False

There is no longer any way to get the file type

as it says at:

firecrest/src/utilities/utilities.py

Line 438 in 028edf3

# follows: https://docs.python.org/3/library/os.html#os.stat_result

what we actually want is enough information that can be used to directly initialise https://docs.python.org/3/library/os.html#os.stat_result, you can do this with %f but not %a

---

I would also note, that /utilities/file is kind of useless, because for this example, it simply returns empty.
So no actual information on what the file type is 😬

[chrisjsewell]

I would note also, it would be great if you could construct stat_result from the return items of utilities/ls:

Currently, you can at least create the mode from the function:

def convert_st_mode(data: dict) -> int:
    # created from `ls -l -A --time-style=+%Y-%m-%dT%H:%M:%S`, e.g.
    # -rw-------  1 username usergroup 57 2021-12-02T10:42:00 file.txt
    # type, permissions, # of links (not recorded), user, group, size, last_modified, name

    ftype = {
        "b": "0060",  # block device
        "c": "0020",  # character device
        "d": "0040",  # directory
        "l": "0120",  # Symbolic link
        "s": "0140",  # Socket.
        "p": "0010",  # FIFO
        "-": "0100",  # Regular file
    }
    p = item["permissions"]
    r = lambda x: 4 if x == "r" else 0  # noqa: E731
    w = lambda x: 2 if x == "w" else 0  # noqa: E731
    x = lambda x: 1 if x == "x" else 0  # noqa: E731
    p_int = (
        ((r(p[0]) + w(p[1]) + x(p[2])) * 100)
        + ((r(p[3]) + w(p[4]) + x(p[5])) * 10)
        + ((r(p[6]) + w(p[7]) + x(p[8])) * 1)
    )
    mode = int(ftype[item["type"]] + str(p_int), 8)

    return mode

[chrisjsewell]

Ah I see it was actually mentioned by @aledabin in the original PR: #133 (comment)

but yeh, unfortunately its wrong for the reasons mentioned above; it needs to encode both permissions and file type

[chrisjsewell]

FYI, this is the actual use case I had in mind for this endpoint 😄 eth-cscs/pyfirecrest#43

[aledabin]

Hi @chrisjsewell , sorry, I misunderstood the whole issue. It will be published and deployed soon.

Regarding the file endpoint it returns the output of the file command, so if the file is empty it returns that. Could that be the reason?

[chrisjsewell]

sorry, I misunderstood the whole issue. It will be published and deployed soon.

Thanks 🙏 and yeh no worries, I didn't realise the implication of the change until trying to use it

Regarding the file endpoint it returns the output of the file command, so if the file is empty it returns that. Could that be the reason?

Well it just seems that thé file command is generally not very useful, from a "programatic" standpoint, because it doesn't really give you any standardised return value to work with

[aledabin]

Hi @chrisjsewell, the fix was included in v1.8.3 and deployed at CSCS last month, I forgot to update here 😞

Regarding file endpoint, I understand your point, maybe it is not that useful to be exposed.

I'll close this issue.
Best,

[chrisjsewell]

No worries, cheers!