Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What happens to watched directories when they are renamed? #165

Closed
devurandom opened this issue Sep 29, 2018 · 14 comments
Closed

What happens to watched directories when they are renamed? #165

devurandom opened this issue Sep 29, 2018 · 14 comments

Comments

@devurandom
Copy link

I am watching a directory (recursively) and also its parent (non-recursively). When I detect a Rename(old, new) event, I tried to unwatch() the old path, but that produced an error (old path is not being watched). The same happened when I tried to unwatch() the new path. Now my question is: How come? What happens to watched directories when they are renamed?

@passcod
Copy link
Member

passcod commented Sep 29, 2018

What platform is this?

@passcod
Copy link
Member

passcod commented Sep 29, 2018

But generally:

@devurandom
Copy link
Author

This would be Linux 4.18.

I assumed notify still holds an fd to the directory and would thus keep watching it even if it moves.

So generally I can assume that when a directory is being watched and then moved, it is not longer being watched and I do not need to undertake any measures to prevent it from being watched? The only thing I do need to do is to watch out for directories that are being moved into locations that I would like to watch?

@passcod
Copy link
Member

passcod commented Sep 29, 2018

So, on Linux, inotify only takes paths as inputs. The kernel does the holding and monitoring, we don't keep FDs ourselves. Other kernel APIs take FDs (like fanotify), but are impractical (such as requiring root and not supporting all events) and also not implemented here.

@passcod
Copy link
Member

passcod commented Sep 29, 2018

So generally I can assume that when a directory is being watched and then moved, it is not longer being watched and I do not need to undertake any measures to prevent it from being watched?

If it's moved out of all watched subtrees, yes. If it's moved between watched subtrees, we watch for that and would add a watch on the "new" path.

The only thing I do need to do is to watch out for directories that are being moved into locations that I would like to watch?

Correct, but also you might not get to know where a directory moved to, because if it moves outside the inotify-watched subtrees, the destination event won't get emitted.

@devurandom
Copy link
Author

devurandom commented Sep 29, 2018

When a directory is moved back into it's original place, it will not automatically be watched by inotify again? (i.e. moving a directory I placed a watch() on to a different location is an implicit unwatch()?)

@passcod
Copy link
Member

passcod commented Sep 29, 2018

By inotify, no. But we catch directory creation within the events we get and add watches for them. (Said events will only give us paths within the subtrees we watch, so it doesn't "escape" the watch root, so to speak.)

@passcod
Copy link
Member

passcod commented Sep 29, 2018

@passcod
Copy link
Member

passcod commented Sep 29, 2018

And I mispoke when I said "Notably, notify will not be watching the new path automatically.". I should have added the caveat "if it's out of our purview". Notify does add watches for directories moved or created into our purview.

@devurandom
Copy link
Author

devurandom commented Sep 29, 2018

I think there might be a misunderstanding. What I mean is:

let (tx, rx) = channel();
let mut watcher = watcher(tx, Duration::from_secs(2)).unwrap();
watcher.watch(".", RecursiveMode::NonRecursive); // Watch the main directory for new folders we would like to watch
watcher.watch("templates", RecursiveMode::Recursive);
loop {
        match rx.recv() {
            Ok(event) => {
                match &event {
                    Create(path) |
                    Write(path) |
                    Rename(old_path, new_path) => {
                        // First situation:
                        // 1. "templates" directory is renamed to "foo.bar"
                        // 2. I receive the event here, but what is the state of `watcher` afterwards?
                        //
                        // Second situation:
                        // 1. "foo.bar" directory is renamed to "templates"
                        // 2. I receive the event here, but what is the state of `watcher` afterwards? Will it automatically watch "templates" recursively again?
                    }}}}

From https://github.com/passcod/notify/blob/master/src/inotify.rs#L232-L270 I gather that for all directories, including those I called watch() on, they will automatically be unwatched by Rust notify, when they are moved out of their original location (i.e. MOVED_FROM). Correct?

@passcod
Copy link
Member

passcod commented Sep 29, 2018

when they are moved out of their original location (i.e. MOVED_FROM), directories will automatically be unwatched by Rust notify, correct?

Yes. The caveats and confusion above is because of the two different watch contexts. In all-recursive contexts, the watches would be re-added and everything would work "as expected". Here, the top context is non-recursive, and rename events are only tracked "from above", so notify doesn't re-add watches. This is possibly a missing feature but I'll have to think more about it.

In more detail:

Because you're watching both from without and within, if the move event is from the watch on ., it will be a MOVED_*, and if the move event is from the watch on template, it will be a MOVE_SELF. As you're watching both, both events will be received, but the contexts of the events are different.

The sequence goes like this:

  1. State here is one non-recursive watch on . and one recursive watch on templates (and subfolders).
  2. Filesystem: rename(templatesfoo.bar)
  3. Inotify: sends a tracked MOVED_FROM: .../templates for the watch on ..
    • Notify receives it and issues an internal remove_watch event on .../templates.
  4. Inotify: sends a tracked MOVED_TO: .../foo.bar for the watch on ..
    • Notify receives it. It's in the context of the watch on ., which is not recursive, so it doesn't issue an internal add_watch event on .../foo.bar.
  5. Inotify: sends an untracked MOVE_SELF: .../templates for the watch on templates.
    • Notify receives it and issues an untracked RENAME: .../templates event.
  6. Notify: Internal events are settled and external events are emitted.
  7. Notify: The debounce layer resolves the tracked MOVED_* events to a single Rename(old, new).
  8. State here is one non-recursive watch on .
  9. Filesystem: rename(foo.bartemplates)
  10. Inotify: sends a tracked MOVED_FROM: .../foo.bar for the watch on ..
    • Notify receives it, but there's no watch on foo.bar so it doesn't do anything special.
  11. Inotify: sends a tracked MOVED_TO: .../templates for the watch on ..
    • Notify receives it. It's in the context of the watch on ., which is not recursive, so it doesn't issue an internal add_watch event on .../templates.
  12. Notify: internal events settle, external events emit, debounce resolves the rename.
  13. State hasn't changed. It's still one non-recursive watch on .

@devurandom
Copy link
Author

Thank you very much for the detailed explanation!

I am in the process of writing an opportunistic watcher, which would be able to watch() directories that do not (yet) exist, by watching their parent directory non-recursively and issuing a recursive watch on the target directory, as soon as it comes into existence.

@passcod passcod closed this as completed Sep 29, 2018
devurandom added a commit to devurandom/notify that referenced this issue Sep 29, 2018
devurandom added a commit to devurandom/notify that referenced this issue Sep 29, 2018
devurandom added a commit to devurandom/notify that referenced this issue Sep 29, 2018
devurandom added a commit to devurandom/notify that referenced this issue Sep 29, 2018
devurandom added a commit to devurandom/notify that referenced this issue Sep 29, 2018
@devurandom
Copy link
Author

Would the OpportunisticWatcher from devurandom/gutenberg@c8975c5 be of interest to Notify? Maybe it could be improved upon, generalised added as an optional addon layered on top of the existing Watchers?

@passcod
Copy link
Member

passcod commented Sep 30, 2018

Current version is feature-frozen ("maintained, not developed") and the next version will have significantly different architecture, so not directly, but I'll keep it in mind!

Mohsen7s pushed a commit to Mohsen7s/arti that referenced this issue Feb 7, 2022
Fixes these messages:

  warning: this URL is not a hyperlink
   --> crates/arti/src/watch_cfg.rs:115:5
    |
115 | /// notify-rs/notify#165 and
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/notify-rs/notify/issues/165>`
    |
    = note: `#[warn(rustdoc::bare_urls)]` on by default
    = note: bare URLs are not automatically turned into clickable links

warning: this URL is not a hyperlink
   --> crates/arti/src/watch_cfg.rs:116:5
    |
116 | /// notify-rs/notify#166 .
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/notify-rs/notify/pull/166>`
    |
    = note: bare URLs are not automatically turned into clickable links
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants