Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add recursive watch #179

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add recursive watch #179

wants to merge 2 commits into from

Conversation

0div
Copy link
Contributor

@0div 0div commented Aug 24, 2024

Description

  • WatchDir now performs a recursive directory traversal and watches recursively
  • Some attention was given to specify which linux folders not to watch in the root folder (see comment for details)
  • When a new directory is created, it is added to the watch list
  • The WatchDirResponse now return absolute paths instead of relative paths to avoid confusions.
  • nit: os.Lstat is used instead of os.Stat to avoid following symbolic links

Test

For now, the tests in js-sdk add some indirect coverage to this implementation

Side note on performance

  • Lowered the number of default linux dirs being watched from ~4000 to ~800 with the regex filter.
  • One call to WatchDir at the root did not increase goroutine count (usually 4)
  • System performance in the container (TL;DR doesn't seem to affect it much)
    • without recursion:
    root@18bf07cd3f0f:/# uptime
    14:02:32 up 0 min,  0 user,  load average: 0.46, 0.10, 0.03
    
    top - 14:02:53 up 0 min,  0 user,  load average: 0.79, 0.20, 0.07
    Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
    %Cpu(s):  0.0 us,  0.1 sy,  0.0 ni, 99.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
    MiB Mem :  15974.8 total,  13557.3 free,   1055.5 used,   1594.0 buff/cache     
    MiB Swap:   1024.0 total,   1024.0 free,      0.0 used.  14919.3 avail Mem 
    • with recursion:
    root@18bf07cd3f0f:/# uptime
    14:03:15 up 1 min,  0 user,  load average: 0.95, 0.29, 0.10
    
    top - 14:03:37 up 1 min,  0 user,  load average: 1.02, 0.36, 0.13
    Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
    %Cpu(s):  0.0 us,  0.0 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
    MiB Mem :  15974.8 total,  13647.2 free,    960.4 used,   1602.5 buff/cache     
    MiB Swap:   1024.0 total,   1024.0 free,      0.0 used.  15014.4 avail Mem 

@ValentaTomas ValentaTomas added the improvement Improvement for current functionality label Aug 30, 2024
@ValentaTomas ValentaTomas marked this pull request as draft September 10, 2024 01:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement for current functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants