Reducing overhead of calling Python code by converting Python bytecode to Java bytecode

[Christopher-Chianelli]

TL;DR:

• There is a significant overhead cost occur when crossing from Java to Python and vice versa
• Converting the Python bytecode to Java bytecode eliminates this cost, at the expense of using a less optimized "Java" Python Interpreter
• On small functions where overhead cost of converting from Java to Python and vice versa take up the majority of the runtime, improvements of 500%->10000% can be seen
• On larger functions, where overhead cost take less of the runtime, converting to CPython and back is better than the Java bytecode as it is more optimized for Python bytecode than the Java interpreter is.

Intro

Inspired by optapy/optapy#2 (comment), I decided to investigate the potential speed improvements that can be obtained by converting the Python bytecode to Java bytecode. OptaPlanner (the Java library OptaPy uses) run small functions (typically attribute getters) many times to calculate score. As such, the overhead cost typically dominates, resulting in score calculation speed at least 20 times slower than Java. I managed to translate most of Python 3.9 bytecode. The result: when translating only Joiners (and still going back to Python to set the attributes and for a filter), 500% improvement in execution time. For instance, for school timetabling:

• Java Score Calculation Speed: 56428/sec
• Python Score Calculation Speed: 436/sec
• Python translated to Java Score Calculation Speed: 2529/sec

How it works

It uses the Python's dis module to disassemble the bytecode of a Python Function. That bytecode is then passed to Java, which is translated to equivalent Java bytecode using ASM. In order to maintain 1-to-1 compatibility, a lot of operations were implemented inefficiently (for instance, integers are stored as PythonInteger, which wraps a BigInteger, and define the method Python uses to perform operations).

Possibilities

Unlike Jython, the translator is designed to work with CPython, and thus can be used with JPype. One possibility is to use it with @JImplements, specified as an additional parameter "translate_to_java_bytecode".

Risks

Unlike Java, most of Python Standard Library is not available as bytecode. As a result, replacements would need to be provided for most of the standard library. Additionally, Python bytecode is not particularly stable. For instance:

• Python 3.8 changed order of key/value on the stack for MAP_ADD
• Python 3.11 completely reworked how exceptions work (bytecode instructions for exception handling were removed, and exceptions now work via dispatch tables (see this CPython issue for details)
• Most Python version add a few opcodes

Try it out

You can find the source of the translator on my python-to-java-translator branch, which also contains the relevant instructions for building and running it.

[Christopher-Chianelli]

@ge0ffrey ^

[Christopher-Chianelli]

Can the foreign function interface take Java objects without incurring the overhead cost for translation/wrapping? I imagine the generated code would be slower due to switching from C -> Java, but I would need to measure it to check. Additionally, Project Panama might affect its performance.

[Thrameos]

You can check out the epypj branch on my copy of the repo. In it I made every CPython call directly callable from within Java (so long as it has one of the standard signatures). The routing cost is simply the time to unpack the array of Java longs and turning it into a pointer, calling CPython, then calling back to Java with the resulting pointer. Thus no actual wrapping on the edges at unless it was something that needed a Python to Java translation on the return.

[Christopher-Chianelli]

The results using the epypj branch is about the same speed as Python: 432/sec.
There were some exceptions in the log (no method PyMethod_New, PyNumber_Add), and I needed to change some code for it to compile (new/renamed/remove fields).

[Christopher-Chianelli]

(I am guessing I need to update my generated classes to call a method (i.e. cannot rely on it being handled automatically, although I did see it was generating classes apparently)

[Thrameos]

The epypj code currently uses the same methods for the Python to Java so I suspect that the speed with be the same for that path. I never got around to trying to rework the proxy code to use the faster marshalling. However, it is also creating direct bindings which can be used if you are decompiling the Python byte code. So it may be able to accelerate what you are trying to do here.