gripe: deprecating fs.exists/existsSync

[KarbonDallas]

After noticing in the node docs that fs.exists and fs.existsSync were going to be deprecated, I took at look at the iojs docs to see that it had in fact been.

This is really annoying and seems like it's based out of the assumption that every developer ever is only checking the existence of a file prior to reading/writing in some async context. While I understand the sentiment behind wanting to avoid 'unnecessary abstractions' or race conditions, this is unnecessary.

Whereas I was previously able to handle the checking of a file's existence with a single boolean variable, I'm now given no other option but to either try/catch or make my code async and listen for error events.

This feels like prescriptivism, as I can't think of a single reason why a stern warning of the potential implications and/or examples of caveats to its use wouldn't have sufficed.

Can anyone help me understand why this was necessary (beyond pushing the all-async-all-the-time paradigm (that doesn't always necessarily apply (particularly in the case of synchronous CLI tooling)))?

Or perhaps can I just submit a PR that un-deprecates this perfectly good functionality?

EDIT: I am happy to provide any additional documentation that is deemed necessary.

[Fishrock123]

This is really annoying and seems like it's based out of the assumption that every developer ever is only checking the existence of a file prior to reading/writing in some async context.

That's not quite how it is.

See fs.exists() in the docs .. it's been replaced by fs.access()

See #114 for more reference.

[Fishrock123]

Reason being: fs.exists() used an API format that was unlike anything else in our API, and could produce results that were unexpected to those who didn't otherwise know.

[KarbonDallas]

fs.access doesn't seem like a replacement given that it also throws when there are any accessibility checks that fail. I don't want/need to do error handling every time I check for a file's existence. I don't think it's fair to call this a replacement.

[Fishrock123]

I'm now given no other option but to either try/catch or make my code async and listen for error events.

That would be correct.

[KarbonDallas]

OK, close the issue before I can even reply? Awesome!

[KarbonDallas]

Deprecation with this kind of logic/motivation is only going to serve to make this platform less hospitable to people doing things outside of what features core is focused on.

I still don't see any reason why thorough documentation isn't an adequate response to any of the concerns that have been mentioned thus far.

[KarbonDallas]

I'm also pretty concerned at the realization that this is how we're treating issues created around valid questions/concerns.

Slapping down an open issue within 20 minutes of its creation is pretty hostile.

[Fishrock123]

Original discussion can be found at nodejs/node-v0.x-archive#8418 & #103

[Fishrock123]

Slapping down an open issue within 20 minutes of its creation is pretty hostile.

The GitHub close issue button is in the same location a cancel button usually is. This isn't uncommon for people to occasionally do. Is the intent not clear since I directly re-opened it? :/

[mscdex]

If you're concerned about having to do a try-catch everywhere for fs.accessSync(), why not just make your own wrapper function(s)?:

var fs = require('fs');
function existsSync(filename) {
  try {
    fs.accessSync(filename);
    return true;
  } catch(ex) {
    return false;
  }
}
function exists(filename, cb) {
  fs.access(filename, cb);
  // or if want exactly the same old API:
 //fs.access(function(err) { cb(err ? false : true); });
}

[KarbonDallas]

or why not leave the wrapper the way it is and let me continue to use fs.existsSync? 😸

[Fishrock123]

perfectly good functionality?

or why not leave the wrapper the way it is and let me continue to use fs.existsSync?

That same thinking can be applied to just adding more and more sugar to core too. There is plenty discussion in the issues I liked above. At minimum, it wasn't particularly perfect, but it wasn't broken either.

[mscdex]

As previously mentioned, the majority use case for fs.exists()/fs.existsSync() was to check if a file existed before opening a file. Removing it and recommending people just use fs.open()/fs.openSync() solves the problems associated with that majority use case.

[KarbonDallas]

@Fishrock123 let's discuss how we can make it good enough to leave in.

[KarbonDallas]

Should I create a new issue, or can we discuss the things that need to be improved on this one?

[bnoordhuis]

This has been discussed to death already. Moving to close.

[KarbonDallas]

@bnoordhuis How about if you're tired of discussing this particular issue: don't involve yourself.

[bnoordhuis]

Kind of hard to do when I get 15 new emails in the 30 minutes I'm away from my computer... at any rate, my point is that you're unlikely to bring up anything that hasn't been brought up before.

[KarbonDallas]

@bnoordhuis understandable. Apologies for contributing to your inbox spam.

I'll take a look at the source, read up on previous conversations, and see what I can do to help fix this.

[sindresorhus]

I made a module for this as there's still genuine use-cases for checking if a file exists without reading the file afterwards.

[ChALkeR]

Btw, what's wrong with fs.access? fs.exists has inconvenient api, for example: petkaantonov/bluebird#418
fs.access in a form fs.access(path, callback) looks to do exactly what fs.exists should have done from the start.
Or am I missing something?

[imyller]

Just stopping by to say that I agree with the point @emilyrose makes here.

It is true that fs.access can fully replace fs.exists and fs.existSync functionality. Especially with wrappers like @sindresorhus kindly implemented. Still many newcomers to the platform will be left looking for simple way to get boolean true/false for file existence as fs.access might not be obvious to them.

I personally think that it is, in some cases, acceptable for io.js core APIs to offer functionality that emphasizes simplicity and rapid-development over formal conventions. fs.exists might belong to that category.

[benjamingr]

@emilyrose the reason it was deprecated is because fs.exists has a broken API - I opened the deprecation issue to either remove or fix this. fs.exists typically also suffers from race condition but you can use fs.access to easily do what you did with exists before or heck - use a userland package there are dozens on NPM.

[benjamingr]

@imyller that sounds like a documentation issue - and a good pull request

[imyller]

@benjamingr that is true. I do agree that fs.access has the correct API and fs.exists has issues, but I think the point of this issue is that there still are solid reasons to have simple file-exists-or-not function returning a boolean.

Additionally, one might want a documentation on why you can create full HTTP server implementation with one-liner, while needing many more lines to check if some local file exists ;)

[rlidwka]

I think the point of this issue is that there still are solid reasons to have simple file-exists-or-not function returning a boolean.

Well you can't even tell if file exists or not with the fs.exists, because in case of an error it'll fail silently even if file does exist. And it does nothing fs.stat can't do.

I don't see a single reason why such an api should exist in the core.

Additionally, one might want a documentation on why you can create full HTTP server implementation with one-liner

Don't worry, the entire http2 community is working on it. In a year or so you won't be able to create http server without getting ssl certificates, and it will surely not be a one-liner anymore

[neuroscr]

Was just about to open an issue on this. Documentation is unclear about access being a replacement. I think the real issue here is how the not found is communicated. If access/stat are to be used for an existent check, then an exception on not found is too severe. Wouldn't returning that it's not FRWX sufficient?

[dzcpy]

fs.exists is really necessary. If the api is broken can we fix it rather than deprecate it? Or otherwise, can we provide an option for fs.access, if the second parameter mode === fs.E_OK then do not throw the error and returns the boolean?

[ChALkeR]

@andyhu Once again: fs.exists doesn't do what one might think it's doing judging by its name, and that is a problem.

Just use fs.access as a replacement, it does exactly the same for all fs.exists usecases (except for having the correct callback signature).

[trevnorris]

Seems at this point the ask is for an API that does:

function doesReallyExistSync(path) {
  try { return !fs.accessSync(path, fs.F_OK) } catch (e) { return false }
}

TBH I understand wanting such a simple API, but the problem is it's not actually that simple. Take the following example where directory b is a symlink back to a:

fs.accessSync('./a' + '/b/a'.repeat(60) + '/c', fs.F_OK)

Which will result in the following:

Error: ELOOP: too many symbolic links encountered, access './a/b/[...]/a/c
    at Error (native)
    at Object.fs.accessSync (fs.js:203:11)

Is node also supposed to add the logic of checking for ELOOP and running the path through fs.realpath() to get around this issue? If so then I'd expect node to do it's best to compensate for any platform specific issues in the API. Here things get a bit hairy, and suddenly an API at first glance was one line becomes more complex. Enough, IMO, to merit its own module (there are commonly used smaller modules out there).

[dfabulich]

I filed PR #7455 to improve the performance of existsSync beyond what's possible in pure JS (by avoiding throwing an ignored exception).

[dfabulich]

@trevnorris Your accessSync example is pretty funky! Luckily, we already know what the right thing to do with that is: existsSync returns false in that case. LGTM, so let's just undeprecate that.

[imyller]

Just for reference I looked up how other languages/runtimes do this:

Python 3 (uses stat, "if stat fails, file is assumed not to exist" -method):

# Does a path exist?
# This is false for dangling symbolic links on systems that support them.
def exists(path):
    """Test whether a path exists.  Returns False for broken symbolic links"""
    try:
        os.stat(path)
    except OSError:
        return False
    return True

Java (JDK 8) (uses stat, "if stat fails, file is assumed not to exist" -method):

JNIEXPORT jint JNICALL
Java_java_io_UnixFileSystem_getBooleanAttributes0(JNIEnv *env, jobject this,
                                                  jobject file)
{
    jint rv = 0;

    WITH_FIELD_PLATFORM_STRING(env, file, ids.path, path) {
        int mode;
        if (statMode(path, &mode)) {
            int fmt = mode & S_IFMT;
            rv = (jint) (java_io_FileSystem_BA_EXISTS
                  | ((fmt == S_IFREG) ? java_io_FileSystem_BA_REGULAR : 0)
                  | ((fmt == S_IFDIR) ? java_io_FileSystem_BA_DIRECTORY : 0));
        }
    } END_PLATFORM_STRING(env, path);
    return rv;
}

Perl (uses stat, -e (exists) op is 1/true if stat has returned any results, false otherwise)

my %op = (
    r => sub { cando($_[0], S_IRUSR, 1) },
    w => sub { cando($_[0], S_IWUSR, 1) },
    x => sub { cando($_[0], S_IXUSR, 1) },
    o => sub { $_[0][4] == $>           },

    R => sub { cando($_[0], S_IRUSR, 0) },
    W => sub { cando($_[0], S_IWUSR, 0) },
    X => sub { cando($_[0], S_IXUSR, 0) },
    O => sub { $_[0][4] == $<           },

    e => sub { 1 }     <--- exists

PHP (uses stat, "if stat fails, file is assumed not to exist" -method):
Snippet from (filestat.c):

/* {{{ proto bool file_exists(string filename)
   Returns true if filename exists */
FileFunction(PHP_FN(file_exists), FS_EXISTS)

...

    case FS_EXISTS:
        RETURN_TRUE;

Go (uses stat with additional library function to test returned error for file existence clues):

if _, err := os.Stat("./thefile.ext"); err != nil {
    if os.IsNotExist(err) {
        // file does not exist
    } else {
        // other error
    }
}

Go language approach is the most interesting: they provide standard stat and then have two separate core library convenience functions for determining if returned error indicates file existence or non-existence os.IsExists(err) and os.IsNotExists(err).

With exception of Go language, most common method for implementing "file exists" test seems to be just running stat and assuming file existence if anything / no error is returned.

[trevnorris]

@dfabulich

we already know what the right thing to do with that is: existsSync returns false in that case.

Not exactly. fs.realpath() should resolve the symbolic links to a path that fs.access() can handle (there's a bug in v6 where that doesn't work, for which I'm working on). So if the user got ELOOP they could then call the operation again with fs.access(fs.realpath(path));. Simply returning false would be technically incorrect.

[imyller]

For those interested:

I've published a userland module fs-exists-nodeback

https://github.com/imyller/node-fs-exists-nodeback

When loaded the module polyfills fs.exists to support both original Node.js callback style and standard error first callback style in a backward compatible way.

This may offer solution to those having issues with the callback(boolean) not working with libraries expecting functions with nodeback standard callback.

[Fishrock123]

I think 7b5ffa4 should fix / address this.

It undeprecates existsSync() and keeps the inconsistent-callback exists() deprecated.