Maya2022 and mayapy/Python 3 - attach_pydevd.send_command

[YKdvd]

I tried the new version of Mayacharm now that it is working on modern PyCharm releases, and got Maya 2020 working with no problem (wonderful, thanks). For Maya 2022, I had the MayaCharm setup pointing to "mayapy.exe", which in 2022 is the Python 3 version; the old Python 2 interpreter is "mayapy2.exe". Mayacharm didn't like this when I tried to Attach to Process:

"C:\Program Files\Autodesk\Maya2022\bin\mayapy.exe" C:\Users\fred.flintstone\AppData\Local\JetBrains\PyCharm2021.1\MayaCharm\pipecore_farm.404b420c\python\attach_pydevd.py --port 53577 --pid 12948 --mcPort 20223 --pydevPath C:\Users\fred.flintstone\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\211.7142.13\plugins\python\helpers\pydev
Traceback (most recent call last):
  File "C:\Users\fred.flintstone\AppData\Local\JetBrains\PyCharm2021.1\MayaCharm\pipecore_farm.404b420c\python\attach_pydevd.py", line 90, in <module>
    main(process_command_line(sys.argv[1:]))
  File "C:\Users\fred.flintstoneAppData\Local\JetBrains\PyCharm2021.1\MayaCharm\pipecore_farm.404b420c\python\attach_pydevd.py", line 86, in main
    send_command(setup['mcPort'], python_code)
  File "C:\Users\fred.flintstone\AppData\Local\JetBrains\PyCharm2021.1\MayaCharm\pipecore_farm.404b420c\python\attach_pydevd.py", line 56, in send_command
    client.sendall('python("' + message.replace(r'"', r'\"') + '")')
TypeError: a bytes-like object is required, not 'str'

Python 3's socket module now uses "bytes" objects, not strings in many cases. You'd probably have to do a bytes() coercion on the sendall() line, and since I'm not sure if socket.sendall() in Python 2.7 accepts bytes, you might have to do an if/else check for Python 2/3 and make the appropriate thing to send for each. I'm not sure if there are any other Python 3 gotchas in the code.

I switched my MayaCharm sdk entry for Maya2022 to "mayapy2.exe" and that should be fine. The choice of .exe in that setting is simply to provide a python to execute the attach_pydevd.py script to talk to the port in the Maya process, right, and it doesn't matter if it is the mayapy2 talking to maya.exe started up in Python 3 mode? But it may be worth making the code Python 2/3 safe, since I believe on MacOS Maya2022 only has the Python 3 mayapy.exe available, and they'll probably drop "mayapy2" on the other platforms eventually.

[YKdvd]

Actually, I can create as setup for Maya 2022 "mayapy2.exe", and it seems to work and breakpoints are hit. But when I go back to the MayaCharm settings, you can see that entry disappear. I think you may be checking your sdk setups when you put up your GUI settings pane, and deleting as invalid any whose executable name (on Windows) isn't exactly "mayapy.exe"? I see a Kotlin constant mayaPyExecutableNameWin = "mayapy.exe" in the repo, but I haven't dug in and tried to find code that might delete entries.

If this is the case, using Maya 2022 will be a pain unless "mayapy2.exe" is allowed or the code is made safe for both Python 2 and 3.

[cmcpasserby]

yeah all of you assumptions there are correct, i will try and find time on the weekend to make it recognize mayapy2.exe properly as well as getting the connect logic working good from both py2 and 3

[cmcpasserby]

fixed the attach script bit to be both python 2 and 3 friendly in 8621f2a
the other changes will be later, since i will need to rethink a few things to support 1 Maya having multiple mayapy's.

[YKdvd]

If MayaCharm starts up attach_pydevd.py with, say mayapy2, but is talking to a Maya2022 running in Python3 mode, does that work? Or does the attach have to be done with the same interpreter that is running in Maya? If the former is okay, than having the attach code work with 2022's mayapy.exe (Python3) may be enough to start with.

If you need to determine which Python version a Maya2022 process is using, to choose from mayapy/mayapy2 SDK entries, it looks like Maya sets the MAYA_PYTHON_VERSION environment variable in its running environment, even if it isn't set at launch, or if you use the -pythonver commandline parameter, or don't specify at all. If you are able to peek into a process's environment to check this as you are evaluating it for an SDK, it looks like you could see if MAYA_PYTHON_VERSION is "2" and use that to discriminate for 2022 processes?

[cmcpasserby]

Yeah where it's hitting the issue, is currently when you try to attach, it searches for running maya instances, for each one it finds, it tries to compute the path to that instances mayapy based on its maya executable path. It then uses that path as both the interpreter to run the attach script on as well as a key to a map of mayapy paths to port numbers so it can figure out which command port is associated with a running maya instance.

That mapping is the state that is saved and where the table data in the setting page comes from, this is also the reason why when you open it, you are seeing your mayapy2 ones disappear since in refreshPythonSdks of ApplicationSettings.kt it tries to update that mapping to make sure it only contains valid values and also add any mayapy's that were added as inpreters to pycharm.

On the weekend i can going to restructure all of that around more, and based it all on the Maya path not the mayapy path and let each maya have multiple interpreters associated with it.

I have not used the new Maya, it still would only be 1 command port right?

[YKdvd]

Yes, for any particular Maya process, it is either Python 2 or Python3, and whatever command port you open up. If you have a Maya2022/2 and a Maya2022/3 running at the same time, you'd have the same problem as with any two Maya.exe processes of the same version that share an SDK choice and not being able to discriminate different ports for the different instances.

Once you have 2022 SDK entries for both mayapy.exe and "mayapy2.exe", then at least you can run a 2022/Py2 and 2022/Py3 at the same time and attach to each, although for a particular PY version only the first instance to open the commandPort could be targeted.

Are you able to get the environment variables of the maya.exe process you are evaluating? Perhaps one could set an environment variable MAYACHARM_DEBUGGING_PORT in Python to the value of the command port when it is opened. Then you could use this value to override the default in the SDK setting, so one could debug multiple Maya instances of the same type? Probably not a big market for that feature though... :)

[cmcpasserby]

could try that, if i did it would be check for that var first in maya's env then default to the old way to get that data if not defined. if there is a system call to get the env something is running it, it could be done, even to get the executable path from a pid in a reliable way i already had to implement it 3 times, and do system calls.
can see that really ugly implementation here.
https://github.com/cmcpasserby/MayaCharm/blob/master/src/main/kotlin/utils/ProcessUtils.kt

[cmcpasserby]

am at work, so can't test on windows right now, but it looks like i can ps eww <PID> on mac to get the env vars assuming i can find something similar on windows and linux as well

[YKdvd]

If you are able to get the environment for the Maya process you are evaluating, there is also MAYA_LOCATION, which should point to something like "C:\Program Files\Autodesk \Maya2022" or "/usr/autodesk/maya2020". Might be useful instead of doing the per-platform exepath-from-pid routine?

BTW, in your MayaAttachDebuggerProvider you use the ProcessInfo provided for the .pid and .executableName attributes. ProcessInfo also seems to have an "executablePath" concept - is that not actually filled in on what you get passed, and you have to do the roundabout through the pid?

[cmcpasserby]

notice in that code that executablePath is private, and i have to access it via getExecutableCannonicalPath, and from my experience working with it, it i have never been able to get the path from it since isPresent is always false on the Optional<String> it returns.

my logic still tries to use it, but always falls back to the ProcessUtils i made.

https://github.com/cmcpasserby/MayaCharm/blob/master/src/main/kotlin/debugattach/MayaAttachDebuggerProvider.kt#L26-L28