LWJGL integration

[Spasi]

Opening this issue to discuss a simple way to integrate DriftFX and LWJGL.

After looking at the current example code, I don't think DriftFX needs to know anything about LWJGL (like #8 suggests). The opposite is a cleaner approach, i.e. the integration work should happen in LWJGL:

• LWJGL creates new bindings for the DriftFX shared library.
• The user adds both DriftFX and LWJGL as dependencies to their project.
• Everything is done in Java, no need for the user to compile native code. The JavaFX setup is done using the DriftFX (Java) API, rendering is done using LWJGL's existing bindings, surface/context management is done using the proposed DriftFX bindings.

This way users could get started much more easily with DriftFX and efxclipse-drift maintainers don't have any extra work to do.

There is just one thing that will simplify the process: It would be nice if DriftFX exposed a C API. LWJGL will then be able to auto-generate the bindings and it would lessen maintenance work going forward. Note that you could still use C++ internally, only the public API would be C. You could also layer the C API on top of the C++ API, assuming there are users that prefer a C++ API.

[redrezo]

i was also planning to use drift in pure java by using lwjgl, but since the first use cases required a native API lwjgl had to wait
i thought i just need a Java API to make the context current, acquire and present textures and use lwjgl for the GL calls
It would be great if the lwjgl side could offer an integration layer
It should be easy to add a C API on top of the existing one
I would appreciate it if you could offer an API proposal that suits your needs since I have almost no experience in designing C APIs

[Renanse]

Would really like to see a solution like that proposed by @Spasi

[Spasi]

Hey @redrezo,

LWJGL now has DriftFX bindings in the driftfx branch¹. See the HelloDriftFX sample and note that there aren't any native methods, the user can integrate JavaFX and OpenGL without writing/building native code. LWJGL exposes a C-like API and internally I was able to use the C++ DriftFX API directly. So this issue can be closed².

Also, there's no extra dependency. The bindings include the DriftFX source³ (both Java and native) and everything is built together. Afaict your license allows this, but please let me know if that's not the case or if you're simply not OK with it.

Now, the problem is that performance is not good. It's actually worse than using PixelWriter/PixelReader. The demo is a port of glxgears, that normally renders in 0.37ms at 4K resolution. With DriftFX, I'm seeing 2.9ms at the default size (600x600 with 2x HiDPI) and 40ms at 4K (i.e. 25fps instead of ~2700fps). The SimpleColorRenderer in this repository also exhibits the same bad scaling with rendering area size.

I enabled verbose logging and it appears that there's lot of texture creation/recreation going on every frame. Is this expected behavior? Btw, I'm testing on Zulu JDK 8, Windows 10, Ryzen 1800X, GeForce GTX 970 with latest drivers. I also tried the 3 different TransferModes and MainMemory was actually a bit faster than NVDXInterop (colors were wrong though, possibly an RGBA/BGRA mix-up somewhere in the pipeline).

Thanks!

---

1. It should build fine on Windows & Linux, haven't tried macOS yet. Build instructions:

   set/export JAVA_HOME=<path to JDK 8 with JavaFX 8>
   vcvarsall amd64 // Windows-only
   ant compile-templates tests // Will take 2-4 minutes, builds everything and runs tests
   ant demo -Dclass=org.lwjgl.demo.driftx.HelloDriftFX
   

2. Converting DriftFX to a C API would still be a good idea, C++ doesn't offer anything interesting to such a simple API.
3. Currently copied, will be changed to a git submodule if this effort goes forward.

[tomsontom]

Great work! We‘ll come back to you on the points you raised and ways to fix them

[redrezo]

Thats a great news @Spasi!
Thanks for your effort!

Regarding your performance observations:
At the moment drift creates for every frame a new texture. Its lifecycle is like creation (acquire) -> client rendering -> queuing it to fx (present) -> fx draws
ing -> deletion
We are planning to implement a reuse mechanism for those textures - so we only need to recreate them if a different size is acquired. This should speed up things a lot.

I'll will have a look at your demo once i find the time

[redrezo]

@Spasi you can give the wip_performance branch a try - it should be way faster

[Spasi]

Hey @redrezo, I tried the wip_performance branch and it's crashing almost immediately when launching the demo:

Stack: [0x000000002b250000,0x000000002b350000],  sp=0x000000002b34e458,  free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [nvoglv64.dll+0x94e949]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.eclipse.fx.drift.internal.GPUSyncUtil.nCreateFence()J+0
j  org.eclipse.fx.drift.internal.GPUSyncUtil.access$000()J+0
j  org.eclipse.fx.drift.internal.GPUSyncUtil$GLSync.<init>()V+5
j  org.eclipse.fx.drift.internal.GPUSyncUtil$GLSync.CreateFence()Lorg/eclipse/fx/drift/internal/GPUSyncUtil$GLSync;+4
j  org.eclipse.fx.drift.internal.QuantumRendererHelper.syncExecuteWithFence(Ljava/util/function/Supplier;)Lorg/eclipse/fx/drift/internal/QuantumRendererHelper$WithFence;+5
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.createTexture(Lcom/sun/prism/Graphics;Lorg/eclipse/fx/drift/internal/Frame;)Lcom/sun/prism/Texture;+155
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.renderFrame(Lcom/sun/prism/Graphics;Lorg/eclipse/fx/drift/internal/Frame;)V+19
j  org.eclipse.fx.drift.impl.NGDriftFXSurface.renderContent(Lcom/sun/prism/Graphics;)V+27
j  com.sun.javafx.sg.prism.NGNode.doRender(Lcom/sun/prism/Graphics;)V+330
j  com.sun.javafx.sg.prism.NGNode.render(Lcom/sun/prism/Graphics;)V+34
j  com.sun.javafx.sg.prism.NGGroup.renderContent(Lcom/sun/prism/Graphics;)V+160
j  com.sun.javafx.sg.prism.NGRegion.renderContent(Lcom/sun/prism/Graphics;)V+111
j  com.sun.javafx.sg.prism.NGNode.doRender(Lcom/sun/prism/Graphics;)V+330
j  com.sun.javafx.sg.prism.NGNode.render(Lcom/sun/prism/Graphics;)V+34
j  com.sun.javafx.tk.quantum.ViewPainter.doPaint(Lcom/sun/prism/Graphics;Lcom/sun/javafx/sg/prism/NodePath;)V+201
j  com.sun.javafx.tk.quantum.ViewPainter.paintImpl(Lcom/sun/prism/Graphics;)V+982
j  com.sun.javafx.tk.quantum.PresentingPainter.run()V+326
j  java.util.concurrent.Executors$RunnableAdapter.call()Ljava/lang/Object;+4
j  java.util.concurrent.FutureTask.runAndReset()Z+47
j  com.sun.javafx.tk.RenderJob.run()V+1
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run()V+8
j  java.lang.Thread.run()V+11

The crash happens in the QuantumRenderer-0 thread (not in QuantumRenderer-DriftFX-Helper). Afaict, there's no OpenGL context current in that thread when glFenceSync is called.

[redrezo]

Sorry @Spasi , i forgot to push the conditional code for windows that prevents the gl fence calls..
The branch should work now for windows as well
also i added a javafx fps counter to the surface itself which can be enabled with -Ddriftfx.showfps=true

[Spasi]

Hey @redrezo, the crash is fixed and performance is much better. Will report exact numbers tomorrow, currently testing over a remote desktop solution which skews performance.

The lwjgl3 driftfx branch has been updated if you'd like to test it yourself.

[Spasi]

One problem with the current API is that there's no way to synchronize the rendering thread with surface presentation. If no throttling is applied, the rendering loop is able to run at 4000+ fps, with no "back-pressure" happening. This appears to be working fine mostly, but when the window is resized to be larger, something fails to keep up. There's stutterring and blank frames and sometimes rendering stops completely for a second or two.

I'm able to reproduce this both with and without -Djavafx.animation.fullspeed=true. When it works fine, with fullspeed enabled and -Ddriftfx.showfps=true, it's getting up to 700-800fps (I think 1000fps is the hard cap for JavaFX), which is not as good as LWJGL3FX but good enough for many use cases.

With the rendering loop throttled to 60fps, it's all nice and smooth at every window size/resolution (including 4K).

[redrezo]

@Spasi
I'm happy to hear that the recent changes improved the performance!
I stumbled upon this back pressure issue while i tried to optimize performance.
I'm working on a swapchain approach. This will also gurantee a constant resource usage compared to pooling. I'm going to support a mailbox and a queue. If your swapchain only holds 2 images in mailbox mode acquire would block until the next image gets available.. this would kind of sync the render thread..
Do you have any other ideas/references how i could provide a sync mechanism to control back pressure?

[tomsontom]

can we close this issue?

[Spasi]

Yes, DriftFX/LWJGL integration is complete.

[Aloento]

Unfortunately, this project has been deprecated.