wakepy WSL support

[fohrloop]

Wakepy current is said to support windows and linux. But how about WSL? Does it work currently, and if not, what method should be used on WSL?

[ccrutchf]

I have interest in this for a project I am working on in Python. I have a similar project written in C# that currently supports WSL. My solution was to compile an exe and call it from WSL since WSL can call Windows exes. Since this exe is executed within Windows and outside of WSL it can call WSL libraries. This seems to be the only way to escape the confines of WSL.

I would be happy to write an implementation of this and submit a pull request. Would probably need to compile the exe as part of a GitHub action and then package it with wakepy or have wakepy download it.

[fohrloop]

Hi @ccrutchf, thanks for leaving a message! I'm very happy to hear you would like to contribute!

Current status of the project

First few words about the state of the repository and the roadmap. I've been working on a big update on wakepy (0.7.x -> 0.8.0) for a while. The update is really a rewrite of all the internals, and I've got to a point where there is one working Method, which is for Linux/GNOME, using D-Bus and org.gnome.SessionManager. The other methods (caffeinate for Mac, SetThreadExecutionState for Windows and possibly org.freedesktop.ScreenSaver for Linux) are yet to be implemented before the 0.8.0 release. So, the current dev branch contains the latest development version which is still a bit work in process.

How to support WSL?

Then, about your suggestion. I really want to add support for WSL on Windows. As a matter of fact, the goal is to support every operating system and desktop environment possible. One thing that I am not so sure about is the addition of an executable for few reasons (1) Added complexity for build process from maintaining perspective (2) Added complexity for users if they want or need to use a source release (3) Adding binaries to a python package requires addition level of trust from the users. Perhaps there are also other reasons.

What I would like to consider first is alternatives which would work with pure python. An by pure python I mean python code which calls something else :) This could be

a) A shared library call. The Window implementation uses the SetThreadExecutionState function from Kernel32.dll. Could we call that directly even from WSL?
b) An executable call. Could we for example call powershell.exe from WSL and could that help?

Even if I am not eager to add a binary to wakepy, I would do it, but perhaps as an additional plugin (like: wakepy-wsl), if we cannot find any other alternatives. So I am fully with you adding WSL support, just after deciding how that should work.

You mentioned that you had compiled an exe. What did the exe call, then? How did it work?

Name for "host system"?

Currently, wakepy has concept of "platform". This means "where wakepy is running." It currently has options: Windows, Linux, MacOS & other. But there is another thing which should be given a name. That is, "what is the host system?". What I mean that is, for example in case of WSL, the platform is WSL (if wakepy is running in a python process inside WSL), and the host system is Windows. In case of cygwin, the platform is Cygwin and host system is Windows. There might be other examples, too. What do you think about the names, is "host_system" good? Or should it be "operating_system" or "current_system"? So, we should give a name to the system which runs the current desktop environment, I think.

What should be done for WSL support?

To get to the point where WSL support can be added, the following thing should be done:

• Finish the tasks for 0.8.0 - I'm working on these
• Decide the name for "host system" -> Issue 150
• Implement the "host system" class in wakepy -> Issue 150
• Add automatic detection of WSL -> Issue 161
• Find a nice way to prevent Windows from going to sleep/idle from within WSL
• Implement the above as a wakepy.Method
• add tests for the new method(s)
• add docs for the new method(s)

And that should be pretty much it. Depending on how fast the other tasks are done, the WSL support could be added in 0.8.0 or in the 0.9.0 release.

Are any of the tasks above interesting to you? Let me know if I can help you in any way!

• Niko

[fohrloop]

I couldn't resist the temptation to try to set the keepawake from WSL. I got a first working PoC. It requires some version of python to be installed on Windows (py.exe must be found and point to real python launcher)

Not saying this is the best way to do it, but at least it is quite simple.

from subprocess import PIPE, run

python_code = r"""
import time
from ctypes import cdll

kernel32_file = r'C:\Windows\System32\kernel32.dll'
kernel32 = cdll.LoadLibrary(kernel32_file)

ES_CONTINUOUS = 0x80000000
ES_SYSTEM_REQUIRED = 0x00000001
ES_DISPLAY_REQUIRED = 0x00000002

result = kernel32.SetThreadExecutionState(ES_CONTINUOUS | ES_SYSTEM_REQUIRED | ES_DISPLAY_REQUIRED)

input()
""".strip().split('\n')

python_code = ';'.join([line for line in python_code if line.strip()])

cmd = ['py.exe', '-c', python_code]
result = run(cmd, stdout=PIPE, stderr=PIPE, universal_newlines=True)

So when this is ran in WSL python process:

• py.exe is launched with a one-liner, which calls the SetThreadExecutionState.
• The input() is just infinite wait. Could be something else, too.

When the py.exe process dies, the SetThreadExecutionState is released automatically. So I don't think this even needs any kind of "clean exit".

I also noticed an experimental package called zugbruecke which could call Windows DLLs, but it is experimental and would add at least two dependencies to wakepy, which I would like to avoid if possible.

[fohrloop]

In addition, it seems to be possible to call DLL's from Powershell:

• https://www.leeholmes.com/load-a-custom-dll-from-powershell/
• https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/add-type?view=powershell-7.4#example-4-call-native-windows-apis

I didn't test this yet, but if it can be made work with the SetThreadExecutionState, it might be more appealing as it requires just powershell.exe, which to my understanding is always available on Windows. The py.exe might not be, although it's pretty probable that someone running python in WSL has installed python also on Windows.

[ccrutchf]

Here is my PowerShell version.

from subprocess import Popen, PIPE
from time import sleep

class KeepAwake:
    def __init__(self) -> None:
        self._powershell: Popen = None

    def __enter__(self):
        self._powershell = Popen(["powershell.exe"], stdin=PIPE, universal_newlines=True)

        code = """$code = @'
[DllImport("kernel32.dll", CharSet = CharSet.Auto,SetLastError = true)]
public static extern void SetThreadExecutionState(uint esFlags);
'@""".strip().split("\n")

        # Add each code block
        for line in code:
            self._write(line)
        # Write a blank line to complete the code block.
        self._write("")

        self._write("$ste = Add-Type -memberDefinition $code -name System -namespace Win32 -passThru")
        self._write("$ES_CONTINUOUS = [uint32]\"0x80000000\"")
        self._write("$ES_SYSTEM_REQUIRED = [uint32]\"0x00000001\"")
        self._write("$ES_DISPLAY_REQUIRED = [uint32]\"0x00000002\"")

        self._write("$ste::SetThreadExecutionState($ES_CONTINUOUS -bor $ES_SYSTEM_REQUIRED)")
        self._powershell.stdin.flush()

    def _write(self, line: str):
        self._powershell.stdin.write(f"{line}\n")

    def __exit__(self, exc_type, exc_val, exc_tb):
        self._write("$ste::SetThreadExecutionState($ES_CONTINUOUS)")
        self._write("exit")
        self._powershell.stdin.flush()

        self._powershell.__exit__(exc_type, exc_val, exc_tb)

awake = KeepAwake()

with awake:
    sleep(30)

To provide my input on some of your other questions:

re: host-system

I feel that this is the best name suggested. Operating system and current system to me imply the operating system running the python code. In the case of WSL this would be Linux when I feel that the value for this property should be Windows. The only other name that comes to mind is something to the effect of "Metal Operating System" to imply the software running directly on the metal and not something in a VM like WSL.

###To revisit the idea of how should WSL be supported
Since WSL2 is a VM, the Windows kernel is not available therefore we cannot directly call SetThreadExecutionState. This is why in my previous work, I had a second executable.

Expectation of PowerShell on the host system.

Currently, the code provided here should run on Vista and later without issue. So this should be a dependency that is fine.

Detecting WSL

To detect WSL, I have had success with the following code

def is_wsl():
    with open("/proc/version", "r") as f:
        kernel_version_str = f.readline().lower()
    
    return "microsoft" in kernel_version_str or "wsl" in kernel_version_str

print(is_wsl())

I am happy to take on any of the tasks listed above. Just let me know.

[fohrloop]

re: wakepy Method for WSL using Powershell & SetThreadExecutionState

Oh, wow, thanks for the really nice implementation! That's much better than the one using python as it has less dependencies. Could you help to understand the code in few places?

• Why we set SetLastError = true?
• What does the $ste = Add-Type -memberDefinition $code -name System -namespace Win32 -passThru do?
• Why there is self._powershell.__exit__(exc_type, exc_val, exc_tb) call? I could not find the __exit__ method in the documentation of Popen. What's the difference to using Popen.terminate()?

We could make thïs a subclass of wakepy.Method, or actually two (one for keep.presenting and one for keep.running mode). I could instruct with this if you want to create a PR yourself. After there is a wakepy.Method implementation, I could test that also manually on my Windows laptop. Do you happen to know will this work on WSL (version 1) or just WSL2 ?

re: host-system

Yeah that could be called host system or "host operating system". I went and closed #150 as I could not come up with a practical use case for listing the separate "system". At least now when I think, just detecting the platform should be enough. If it's not in some special case, the platform name can have the system appended to it or something.

re: detecting WSL

Thanks for the code! That looks very promising! I created a task for implementing this: #161. What is still a bit unclear is that if we need to separate detecting WSL1 vs WSL2 (would some possible method now or in the future require WSL1 / WSL2?). If so, there is a long thread at https://github.com/microsoft/WSL/issues -> Issue 4071 with possible ways to do it.

[fohrloop]

Next good candidate task for this is to (1) decide is WSL1 / WSL2 should be detected separately and (2) implement the WSL detection logic (issue 161)). If you want to grab that one, I am happy to help. The DEV.md has some instructions, but I'm not sure if they're perfect. If you have any questions on anything related to contributing, I would be happy to update the DEV.md or write separate CONTRIBUTING.md.

[ccrutchf]

re: wakepy Method for WSL using Powershell & SetThreadExecutionState

• SetLastError will let us look up the result of the function. We are not using in the snippet above, but probably should.
• The code block in the PowerShell code is C#. So the Add-Type call sets it up as something we can call within PowerShell.
• It may be better to call Popen.terminate. Since I am using this with a context manager, I wanted to maintain the Popen context manager semantics. I have picked up this since the standard library also does it in places (or perhaps, used to). Here is an example: https://hg.python.org/cpython/file/767fd62b59a9/Lib/multiprocessing/synchronize.py#l99

re: next steps

The code to block sleep should be the same as both WSL1 and WSL2 can call Windows exes. I may try to whip up a PR this weekend and send it your way.

[fohrloop]

Hi @ccrutchf, great to hear back from you! The wakepy 0.8.0 seems really close, and I have basically just documentation work and maybe some cleanups to be done.

Thanks for the explanation! I'm not too familiar with C#, so good to have your contribution!

Btw. I discovered a bug in the current Windows SetThreadExecutionState method. It is unlikely to affect most of the users as in requires two context managers / mode activators to be used at least partly same time inside a single thread. See: #167 The proposed method here open should not have that problem as the SetThreadExecutionState is called in a child process. Maybe the Windows side wakepy Method could also use Popen.

I also noticed that with some Windows options (Screen Lock disabled or without password), the SetThreadExecutionState based "keep.running" mode actually prevents also screen lock. The same will be true with this method for WSL. More on that here: #169 What are your thoughts on it?

About the PR: I'm not sure if you meant to have PR for the WSL method or some other parts (or both), but all PRs are welcome :) In case you're writing the wakepy.Method for WSL, a good starting point is to look at wakepy/methods/windows.py and create a corresponding wakepy/methods/wsl.py

[ccrutchf]

I have submitted my first swag at this issue and #161. We could also use that PR to address #167. I still want to update this with unit tests, but am trying to figure out how best to approach them.

I agree with #169 as bug. I think it would make sense for wakepy to lock the screen on Windows as that is the behavior the end user would expect. I'll provide my feedback on that issue.

[ccrutchf]

While reading through the issue more thoroughly, I'm still concerned that the functions will not match the user's expectations, even when trying to implement the locking functionality.

PowerToys sets the precedent of not worrying about locking:
https://github.com/microsoft/PowerToys/blob/c406a15099584a069374ab5531e1f711c5b53315/src/modules/awake/Awake/Core/Manager.cs

They even have an issue about this:
https://github.com/microsoft/PowerToys/issues

Search 27178.

[fohrloop]

Thank you for checking it out and providing your view! I continued the discussion about the #169 on the issue itself.

Thank you again for the PR! Really glad that you've written such a nice implementation to support WSL! I commented already on the PR that it would be nice if that could be used to tackle the issue 167 regarding the windows method.

[fohrloop]

The status of this ticket:

• There was PR #190 from @ccrutchf which has quite nice implementation for this. I reviewed it on January 28th but there has been no further activities on it. When dev branch was removed it automatically also closed the PR 190. I have tried to ping him on the PR and asked what's his opinion about the future of the PR by email on 2nd June. Now I'll wait for some time if I get a response so we know how to proceed with this. It's holiday season I'm expecting that a response could take some weeks.
• Needs still tests
• Needs still docs
• TBD: Could the Windows SetThreadExecutionState use exactly same method as the WSL?

After the WSL support is merged to main (with docs and tests), it will be released probably immediately to PyPI.