Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use library current-processes to generate the list of processes #35

Open
branneman opened this issue Jun 19, 2014 · 20 comments
Open

Use library current-processes to generate the list of processes #35

branneman opened this issue Jun 19, 2014 · 20 comments

Comments

@branneman
Copy link

https://www.npmjs.org/package/current-processes

If there's anything you need help with, let me know.

@MrRio
Copy link
Owner

MrRio commented Jun 19, 2014

Thanks very much :)

@MrRio
Copy link
Owner

MrRio commented Jun 19, 2014

It's getting there :) I just need to get at memory as a percentage to keep current behaviour if that's possible.

@branneman
Copy link
Author

Are the native os.totalmem() and os.freemem() not enough?

MrRio added a commit that referenced this issue Jun 19, 2014
@MrRio
Copy link
Owner

MrRio commented Jun 19, 2014

screen shot 2014-06-19 at 21 03 01

It's probably a bug in my logic - or it's a different way of measuring the memory usage, but that's the 'master' branch on the left, and the 'feature-current-process' branch on the right. Getting very low memory usage numbers by comparison.

@branneman
Copy link
Author

My advise would be to check your ps manually. I think its not very realistic for a Chrome Helper to use 25% of memory, right?

@MrRio
Copy link
Owner

MrRio commented Jun 19, 2014

Looks ok to me. It's all 35 chrome helper processes added up (one for each tab). This matches up with activity monitor and ps. 

vtop adds processes that are alike together. 

Sent on the run. Apologies for brevity.

On Thu, Jun 19, 2014 at 9:10 PM, Bran van der Meer
notifications@github.com wrote:

My advise would be to check your ps manually. I think its not very realistic for a Chrome Helper to use 25% of memory, right?

Reply to this email directly or view it on GitHub:
#35 (comment)

@branneman
Copy link
Author

Hm.. let me check. The linux adapter uses ps -A -o pid,vsz,pcpu,comm, which is simply ran through a parseInt() here. Maybe I should do some more parsing on it. Hmmm.

@branneman
Copy link
Author

The description of the vsz column in ps:

virtual memory size of the process in KiB (1024-byte units). Device mappings are currently excluded; this is subject to change. (alias vsize).

From: http://unixhelp.ed.ac.uk/CGI/man-cgi?ps

@branneman
Copy link
Author

How did you calculate memory in the old vtop?

@MrRio
Copy link
Owner

MrRio commented Jun 19, 2014

ps -ewwwo %cpu,%mem,comm

Just used the %mem value from that

@branneman
Copy link
Author

Hmm.. vsz seems the wrong number. And getting the actual memory usage of a process seems not possible with ps :(

http://virtualthreads.blogspot.nl/2006/02/understanding-memory-usage-on-linux.html

@branneman
Copy link
Author

Memory

Ok, so this is a little more complex than I expected. Basically, there's 2 numbers that are interesting when it comes to memory usage of a process, virtual memory and private memory.

Virtual Memory

Virtual memory is the amount of memory a process uses including all shared libraries. Those libraries are shared with other processes, but are only using memory once. If you've got 2 processes both using only 5mb of private memory, and they're using the same shared library which uses 10mb, then the virtual memory for both processes would be 15mb, a total of 30mb. But the actual total memory used is 20mb, since the shared library is only loaded to memory once. If you're doing calculations with virtual memory, total memory usage could exceed 100%, which is weird.

Private Memory

This is the amount of memory the process executable's source code plus the variables in it's private memory space actually takes. Shared libraries can't access memory inside the process that's using the shared library, hence it's called private.

And now the bad news: ps can not output private memory. The windows-equivalent can. So for unix/linux I would need to call a tool like pmap, to get the right private memory.

CPU

And the same goes for cpu usage, the percentage is actually a guesstimate: the OS just gives us the cputime, which is the amount of seconds a process has used the cpu, since the start of the process. So if a process is running a total of 10 seconds, and it used the cpu for 1 second total in that time, it's using 10% of cpu, when 10 seconds are measured. But if we're measuring a 5 second period, we would say it's using 20% of cpu. So I think I'm going to make the underlying numbers available as well.

Conclusion

To account for all these situations, and give the user as much flexibility as possible, I should expose all these variables. Like this:

{
  pid: 4432,
  name: 'chrome',
  mem: {
    virtual: 78923,
    private: 56572,
    percentage: private / os.totalmem() * 100
  },
  cpu: {
    runtime: 65,
    cputime: 10,
    usage: cputime / runtime * 100
  }
}

@MrRio
Copy link
Owner

MrRio commented Jun 20, 2014

Shared memory really does complicate this. OSX has a concept of 'memory pressure' which is something I've been looking into.

@MrRio
Copy link
Owner

MrRio commented Jun 20, 2014

This looks really good, makes the current-processes lib very useful

@branneman
Copy link
Author

Ok, so virtual and private memory are available, you can track progress here: branneman/current-processes#4 - You could use the repo directly already if you want to test with it.

The cpu times are not yet available, will npm publish after those are done as well.

@branneman
Copy link
Author

It's ready! Turns out calculating the cpu usage manually is undoable cross-platform, so it's using the one provided by ps and wmic, which is the same as on tools like top and Taskmanager on Windows.

The new version is v0.2.0

@branneman
Copy link
Author

@MrRio - When do you think you can continue implementing the current-processes library? Or do you need/want some help with it?

@MrRio
Copy link
Owner

MrRio commented Jul 3, 2014

A hand on the feature-current-process branch would be amazing

@soyuka
Copy link

soyuka commented Jul 4, 2014

Check my https://www.npmjs.org/package/pidusage code to get pcpu and memory. I've made it work on darwin, freebsd, solaris (with ps), unix by using /proc and windows with wmic.exe (currently in a beta stage).

In the case of /proc I've commented with a lot of usefull links that might help you for this task: https://github.com/soyuka/pidusage/blob/master/lib/stats.js#L29.

About the pcpu, you'll need to keep an history if you want accuracy.
See this really interesting stackexchange about top and ps: http://unix.stackexchange.com/a/58541/66995

@mnpenner
Copy link

Memory is still off by about a factor of 2 on Ubuntu http://i.imgur.com/XcPqRJb.png

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants