Name field sometimes None on Windows

[alberthdev]

Sometimes, for some odd reason, the psutil returns a process with a name of None.

For instance, using this code:

import psutil
import time

def fetchProcesses():
    for p in psutil.process_iter():
        try:
            p.dict = p.as_dict(['pid', 'name', 'cpu_percent'])

            if p.dict['cpu_percent'] > 70:
                print p.dict
        except psutil.NoSuchProcess:
            pass
fetchProcesses()
time.sleep(0.5)
fetchProcesses()

I'll get output like this:

{'name': u'PlexScriptHost.exe', 'cpu_percent': 97.4}
{'name': u'PlexDlnaServer.exe', 'cpu_percent': 97.4}
{'name': None, 'cpu_percent': 85.2}

So I then tried playing with win32api a bit to grab the file name:

def fetchProcesses():
    for p in psutil.process_iter():
        try:
            p.dict = p.as_dict(['pid', 'name', 'cpu_percent'])

            if p.dict['cpu_percent'] > 70:
                if p.dict['name'] == None:
                    print "DEBUG: need to correct process name"
                    print "DEBUG: opening PID: %i" % p.dict['pid']
                    pHandle = win32api.OpenProcess(
                        win32con.PROCESS_VM_READ | win32con.PROCESS_QUERY_INFORMATION,
                        0, p.dict['pid'])
                    if pHandle:
                        filen = win32process.GetModuleFileNameEx(pHandle, 0)
                        print "Got:", filen

Which resulted in this error:

{'pid': 7272, 'name': u'PlexScriptHost.exe', 'cpu_percent': 92.3}
{'pid': 9360, 'name': u'PlexDlnaServer.exe', 'cpu_percent': 96.5}
DEBUG: need to correct process name
DEBUG: opening PID: 23320
Traceback (most recent call last):
  File "monitor.py", line 27, in <module>
    fetchProcesses()
  File "monitor.py", line 17, in fetchProcesses
    0, p.dict['pid'])
pywintypes.error: (5, 'OpenProcess', 'Access is denied.')

Supposedly, you can do some Windows magic to ask for special privileges for making OpenProcess work, but it seemed a bit too hacky at the time, so I avoided it. (You may opt to try this in the future, though! OpenProcessToken is the starting point.)

I took a cursory look at your code, and it looks like you use OpenProcess as well for process information. (I might be wrong about how you use it... feel free to correct me!)

To work around the issue, I wrote this:

import win32com  
from win32com.client import GetObject  
WMI = GetObject('winmgmts:')
processes = WMI.InstancesOf('Win32_Process')

for process in processes:
    try:
        print('PID: ', process.Properties_('ProcessID').Value, 'process: ', process.Properties_('Name').Value)
    except:
        pass

This does the trick for me, yielding:

...
('process: ', u'blender-app.exe', 'PID: ', 23320)
...

For now, using the above code will be a workaround in case this happens.

It looks like using WMI might be an interesting vector to try. (It might even potentially fix #549... maybe.) What do you think?

[giampaolo]

Hmmm... interesting. I've never seen None as a result value.
For reference, the way the process name is determined right now works like this:

• OpenHandle -> GetProcessImageFileNameW (which returns the exe) -> os.path.basename(exe):

  psutil/psutil/_psutil_windows.c

  Line 616 in 3916125

  /*

• if that fails for permission errors we try CreateToolhelp32Snapshot:

  psutil/psutil/_psutil_windows.c

  Line 647 in 3916125

  psutil_proc_name(PyObject *self, PyObject *args) {

The reason we have two implementations for the same thing is because the first method is faster (so we attempt it first) but it fails with AccessDenied for processes owned by another user in which case we fall back on using method #2.

The code you pasted shows we can potentially use a third method to determine the process name, that is OpenHandle -> GetModuleFileNameEx so I'll try to see how that works. My guess is that it will be faster than using CreateToolhelp32Snapshot.

I'm not sure where None is coming from but I'm more keen to think it comes from CreateToolhelp32Snapshot. Can you verify that? What you should do is put some debugging prints in here:

psutil/psutil/_pswindows.py

Line 276 in 2c76acd

def name(self):

...then reinstall from sources. If you don't have Visual Studio (hence you cannot reinstall) you can figure out where _pswindows.py is located with:

>>> import psutil._pswindows
>>> psutil._pswindows  # this will give you the path
<module path ...>

...modify it in place, then re-run your code.

[giampaolo]

For future reference: I've tried GetModuleFileNameEx but we get the same AccessDenied exceptions we currently get with GetProcessImageFileNameW (the error actually originates from OpenProcess) therefore we won't have to add a third method for determining the process name.

As for the issue at hand, I still can't figure out where None might be coming from so feel free to update the ticket in case you get the chance to investigate this.

[eight04]

I think that is because self.exe() will raise AccessDenied error, but only OSError get caught.

psutil/psutil/_pswindows.py

Line 290 in 3916125

return os.path.basename(self.exe())

I guess there are more similar issues since my program stop working after upgraded to psutil v3.0.1.

[giampaolo]

Ahhh wait. I didn't realize you were using as_dict (I thought you were using .exe() instead).
Having None as the exe value is expected because that's what you'll get with as_dict in case of AccessDenied exceptions. If you want to change that do:

p.as_dict(['pid', 'name', 'cpu_percent'], ad_value="?")`

[giampaolo]

...as for the other errors after upgrading to psutil 3.0.1: I removed a lot of deprecated APIs in 3.x so it's likely you'll get into AttributeErrors. http://grodola.blogspot.com/2015/06/psutil-30.html has instructions on how to port existent code.

[alberthdev]

Sorry for my absence on this issue! This bug is actually tricky to reproduce - I got it when I was running Blender and rendering with NVIDIA CUDA - somehow, psutil was unable to pick up the blender-app.exe render process, while my (unelevated) task manager was.

If you have a NVIDIA card and some spare time, you can try to reproduce this bug by downloading Blender, enabling GPU rendering, opening a Cycles scene, and rendering it. While rendering, try to use as_dict() to fetch the process list. For that strange instance, I was unable to get "blender-app.exe" - the process existed, but the name was left blank. Any ideas on why that may be the case?

I definitely would like to test this further - I have VS/VC++ installed, so it's easy for me to rebuild. When I get a chance (sometime late this month or next month), I'll try to reproduce the bug again, and when I do, get some (hopefully helpful) debugging info for you!

[giampaolo]

I think there's no bug here. You're getting None because that's what you're expected to get when as_dict bumps into a process info (the exe in this case) raising AccessDenied. If you use p.exe() instead of p.as_dict() you'll see the exception being raised. There's nothing you can do about that. Task manager and process explorer are able to get that info because they run as Windows services.

[alberthdev]

The bug isn't that it's returning None due to AccessDenied (that's normal behavior), it's that it could use an alternative method of fetching the process name without requiring any elevation, thus providing another option if we get an AccessDenied. (And of course, if all fails, then we return None.)

In the code snip above, I was able to successfully access Windows services (WMI/winmgmts) to determine the PID and process name. Depending on the OS, there may be other alternate APIs that can access this info as well. (Like you said, Task Manager/ProcExp may hinge on various services for their information... and if they can, it might be possible for you, too!)

[giampaolo]

Installing psutil as a windows service is not an option (assuming it's even possible). We already extensively discussed this possibility.
Can you run this and paste the output?

import psutil

for p in psutil.process_iter():
    if p.pid in (0, 4):
        continue
    print p.pid
    try:
        print repr(psutil._psplatform.cext.proc_exe(p.pid))
    except Exception as exc:
        print exc
    try:
        print repr(psutil._psplatform.cext.proc_name(p.pid))
    except Exception as exc:
        print exc
    print 

[Hellowlol]

print repr(psutil._psplatform.cext.proc_name(p.pid)) this lines showed all the process names on my system instead of None. when using as_dict()

Seems to be something funky with as_dict and name. See my example below.

for p in psutil.process_iter():
    try:
        d = p.as_dict(['name'])
        print d['name'], psutil._psplatform.cext.proc_name(p.pid)
    except Exception as e:
        print e

python 2.7.4 windows 7 home, psutil 3.0.1
Output: https://gist.github.com/Hellowlol/3cd2e87881601dcd2c46

[giampaolo]

You were right: there was a (serious) bug preventing to retrieve process name for PIDs owned by another user. From now on psutil will be able to retrieve name for all processes. See 1c27b34.