[WINDOWS] Uncaught exception in exe() method ?

[EccoTheFlintstone]

The exe() method raises an exception (uncaught/unhandled exception type?) when querying special processes

python launched as admin (in an elevated CMD prompt) on windows 10, getting info on special process "Memory Compression" (pid 1956)

>>> import psutil
>>> psutil.__version__
'5.6.2'
>>> psutil.Process(1956)
psutil.Process(pid=1956, name='Memory Compression', started='2019-04-16 09:09:04')
>>> psutil.Process(1956).exe()
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 774, in exe
    exe = cext.proc_exe(self.pid)
PermissionError: [WinError 31] A device attached to the system is not functioning: '(originated from QueryFullProcessImageNameW)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 773, in exe
    exe = self._proc.exe()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
psutil.AccessDenied: psutil.AccessDenied (pid=1956, name='Memory Compression')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 790, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=True)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 775, in exe
    return guess_it(fallback=err)
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 757, in guess_it
    cmdline = self.cmdline()
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 790, in cmdline
    return self._proc.cmdline()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 708, in convert_oserror
    raise exc
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 793, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=False)
OSError: [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'

This fails the same way on psutil >= 5.6.0 (probably due to using QueryFullProcessImageNameW now or not catching the right error code)

It "works" (doesn't crash) on 5.5.1

>>> import psutil
>>> psutil.__version__
'5.5.1'
>>> psutil.Process(1956).exe()
''

This kind of code is broken because of this exception as getting the 'exe' field crashes the whole function call:

for p in psutil.process_iter(attrs=['pid', 'ppid', 'exe']):
	...

Not sure if this is expected or if the exception caught is currently unhandled

As a normal user (not admin), getting the exe() field of "normal" process which we don't have the rights to get produces the following (we get the psutil.AccessDenied exception):

>>> import psutil
>>> psutil.Process(404)
psutil.Process(pid=404, name='csrss.exe', started='2019-04-16 09:09:04')
>>> psutil.Process(404).exe()
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 774, in exe
    exe = cext.proc_exe(self.pid)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 773, in exe
    exe = self._proc.exe()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
psutil.AccessDenied: psutil.AccessDenied (pid=404)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 790, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=True)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 793, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=False)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 775, in exe
    return guess_it(fallback=err)
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 757, in guess_it
    cmdline = self.cmdline()
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 790, in cmdline
    return self._proc.cmdline()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
psutil.AccessDenied: psutil.AccessDenied (pid=404)

[giampaolo]

Mmmm... it seems there are different things at play here. This one:

  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 793, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=False)
OSError: [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'

If the PID did not exist at this moment in time then we should convert that (WinError 1168) into NoSuchProcess. Can you reproduce this case reliably (and verify)?

As for the second exception (AccessDenied) that is expected and "fine" unless it breaks as_dict() call (but it shouldn't).

[EccoTheFlintstone]

Yes, it can be reproduced reliably :

PID 1732 is "Memory Compression", a "pseudo" process maintained by windows 10 (another one is "Registry" on windows 10) :

>>> import psutil
>>> psutil.__version__
'5.6.2'
>>> psutil.Process(1732)
psutil.Process(pid=1732, name='Memory Compression', started='2019-05-06 01:21:00')
>>> psutil.Process(1732).exe()
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 774, in exe
    exe = cext.proc_exe(self.pid)
PermissionError: [WinError 31] A device attached to the system is not functioning: '(originated from QueryFullProcessImageNameW)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 773, in exe
    exe = self._proc.exe()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
psutil.AccessDenied: psutil.AccessDenied (pid=1732)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 790, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=True)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 775, in exe
    return guess_it(fallback=err)
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 757, in guess_it
    cmdline = self.cmdline()
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 790, in cmdline
    return self._proc.cmdline()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 708, in convert_oserror
    raise exc
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 793, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=False)
OSError: [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'

Note the first exception raised :

exe = cext.proc_exe(self.pid)
PermissionError: [WinError 31] A device attached to the system is not functioning: '(originated from QueryFullProcessImageNameW)'

---

As for the second one, from what I understand :
as_dict is also broken in this case :

>>> a=psutil.Process(1732)
>>> a.as_dict()
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 790, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=True)
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 638, in as_dict
    ret = meth()
  File "C:\Python36\lib\site-packages\psutil\__init__.py", line 790, in cmdline
    return self._proc.cmdline()
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 718, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 708, in convert_oserror
    raise exc
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 716, in wrapper
    return fun(self, *args, **kwargs)
  File "C:\Python36\lib\site-packages\psutil\_pswindows.py", line 793, in cmdline
    ret = cext.proc_cmdline(self.pid, use_peb=False)
OSError: [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'

==> QueryFullProcessImageNameW and NtQueryInformationProcess seem to fail for those pseudo processes and the specific error is not handled ?

[giampaolo]

PID 1732 is "Memory Compression", a "pseudo" process maintained by windows 10 (another one is "Registry" on windows 10) :

Oh! I had no idea such a process existed. That raises different questions. Are cmdline() and exe() the only methods misbehaving? Quick way to test:

import psutil

p = psutil.Process()
for name in psutil._as_dict_attrnames:
    meth = getattr(p, name)
    if callable(meth):
        try:
            meth()
        except psutil.Error: 
            pass
        except Exception as err:
            print(name)
            print(err)

As for the second one, from what I understand: as_dict is also broken in this case

Definitively. Any method raising anything else other than AccessDenied and NoSuchProcess should be considered a bug.

[EccoTheFlintstone]

Apparently yes, only exe() and cmdline() misbehave :
...

...
ERROR in cmdline : [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'
...
ERROR in exe : [WinError 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'
...

(last exception is in NtQueryInformationProcess but we should also handle errors in QueryFullProcessImageNameW)

[giampaolo]

Do you know how I can create such a process myself (I'd like to try it out - am also curious what ProcessHacker does about this). If only cmdline() and exe() are affected then I guess the proper fix should be catching errno 1168 / ERROR_NOT_FOUND in C and return AccessDenied("some message");.

[EccoTheFlintstone]

those processes (Registry and/or Memory Compression) are out of the box on a windows 10 system
you would just need one VM with this sytem to test it out

In process explorer (from sysinternals) :
Registry
path : [A device attached to the system is not functioning.]
commandline : empty
image file : empty (no real process behind)

Memory Compression : same

[giampaolo]

I have those processes (win 10) but they work fine for me. I suppose if you run tests you bump into a failure right? (that's good) If you're up for it feel free to provide a PR, else I will do it sometime later.

[EccoTheFlintstone]

Try with an elevated command prompt (where you can actually open a handle on the process but the API call fail). I think it's working fine with a standard user (with no admin rights)

[bassplayer12]

Hi,

I encountered the same error:

raceback (most recent call last):
File "C:\wptagent\wptagent.py", line 110, in run_testing
self.run_single_test()
File "C:\wptagent\wptagent.py", line 189, in run_single_test
browser.stop(self.job, self.task)
File "C:\wptagent\internal\chrome_desktop.py", line 136, in stop
DesktopBrowser.stop(self, job, task)
File "C:\wptagent\internal\desktop_browser.py", line 284, in stop
kill_all(os.path.basename(self.path), False)
File "C:\wptagent\internal\os_util.py", line 24, in kill_all
wait_for_all(exe, timeout)
File "C:\wptagent\internal\os_util.py", line 32, in wait_for_all
pinfo = proc.as_dict(attrs=['pid', 'name', 'exe'])
File "C:\Python27\lib\site-packages\psutil_init_.py", line 638, in as_dict
ret = meth()
File "C:\Python27\lib\site-packages\psutil_init_.py", line 775, in exe
return guess_it(fallback=err)
File "C:\Python27\lib\site-packages\psutil_init_.py", line 757, in guess_it
cmdline = self.cmdline()
File "C:\Python27\lib\site-packages\psutil_init_.py", line 790, in cmdline
return self._proc.cmdline()
File "C:\Python27\lib\site-packages\psutil_pswindows.py", line 718, in wrapper
raise convert_oserror(err, pid=self.pid, name=self._name)
File "C:\Python27\lib\site-packages\psutil_pswindows.py", line 708, in convert_oserror
raise exc
WindowsError: [Error 1168] Element not found: '(originated from NtQueryInformationProcess(ProcessBasicInformation))'

I just want to ask about the resolution or workaround about this.

Thank you