Custom FrameBuffer implementation

The Problem

The default FrameBuffer of libGDX doesn't support nested framebuffers. This leads to weird bugs, when one framebuffer is used inside another one. For instance, check out this example render() method:

fbo0.begin();
scene.render(); // this scene is rendered into fbo0 

fbo1.begin();
scene.render(); // this scene is rendered into fbo1
fbo1.end();

scene.render(); // this scene should be rendered into fbo0 again; 
                // but the default FrameBuffer breaks here and 
                // stuff is rendered on the actual screen instead

fbo0.end();
scene.render(); // this scene is rendered on the actual screen

Generally, framebuffers work the following way:

• On begin() OpenGL is set to draw to the framebuffer by binding the buffers handle (Gdx.gl20.glBindFramebuffer(GL20.GL_FRAMEBUFFER, framebufferHandle)).
• On end() the default framebuffer is re-bound (Gdx.gl20.glBindFramebuffer(GL20.GL_FRAMEBUFFER, defaultFramebufferHandle)) and subsequent render passes now happen directly on the screen again.

When framebuffers are nested, this leads to the following bug: In the example above, in the fbo1.end()-call the libGDX implementation of FrameBuffer would bind the default framebuffer again. But actually fbo0 should be bound, as the rendering is supposed to happen in that framebuffer. To fix this behaviour, the NestableFrameBuffer tracks what framebuffer was bound before it began to draw. So if end() is called on a NestedFrameBuffer, the previously bound buffer can be re-bound again.

Consequences for Users of libgdx-screenmanager

For this reason, any framebuffer that is used inside of a screen has to be a NestableFrameBuffer, otherwise an IllegalStateException is thrown:

java.lang.IllegalStateException: The currently bound framebuffer (0) doesn't match this one. Make sure the nested framebuffers are closed in the same order they were opened in!
    at de.damios.guacamole.gdx.graphics.NestableFrameBuffer.end(NestableFrameBuffer.java:151)
    at de.damios.guacamole.gdx.graphics.NestableFrameBuffer.end(NestableFrameBuffer.java:126)
    at de.eskalon.commons.utils.ScreenFboUtils.screenToTexture(ScreenFboUtils.java:49)
    at de.eskalon.commons.screen.ScreenManager.render(ScreenManager.java:309)
    at de.eskalon.commons.screen.ScreenManager.render(ScreenManager.java:298)
    at de.eskalon.commons.core.ManagedGame.render(ManagedGame.java:75)

NestableFrameBuffers are provided by the guacamole library, which is automatically included with libgdx-screenmanager.

External Libraries

If you are using any external libraries, you need to ensure that they are using NestableFrameBuffers as well or rebind the outer framebuffer again manually. Some of the more popular libraries, where this is necessary, include:

• box2dlights: You can either use custom LightMap and RayHandler implementations (see here and here) or, instead of calling rayHandler.updateAndRender(), do something like this:

  // Update the lights
  rayHandler.update();
  
  // Save the current FBO status
  int previousFBOHandle = GLUtils.getBoundFboHandle(); // GLUtils is part of https://github.com/crykn/guacamole
  int[] previousViewport = GLUtils.getViewport();
  
  // Prepare the lights
  rayHandler.prepareRender();
   	
  // Restore the FBO status
  Gdx.gl20.glBindFramebuffer(GL20.GL_FRAMEBUFFER, previousFBOHandle);
  Gdx.gl20.glViewport(previousViewport[0], previousViewport[1], previousViewport[2], previousViewport[3]);
  
  // Render to the screen
  rayHandler.renderOnly();

• gdx-gltf: instead of calling sceneManager.render(), something like this may work:

  // Save the current FBO status
  int previousFBOHandle = GLUtils.getBoundFboHandle(); // GLUtils is part of https://github.com/crykn/guacamole
  int[] previousViewport = GLUtils.getViewport();
  
  // Prepare the scene
  sceneManager.renderShadows();
  sceneManager.renderMirror();
  sceneManager.renderTransmission();
  
  // Restore the FBO status
  Gdx.gl20.glBindFramebuffer(GL20.GL_FRAMEBUFFER, previousFBOHandle);
  Gdx.gl20.glViewport(previousViewport[0], previousViewport[1], previousViewport[2], previousViewport[3]);
  
  // Render to the screen
  sceneManager.renderColors();

• gdx-vfx's VfxFrameBuffers are compatible with libgdx-screenmanager, but you might need to ensure that VfxManager is using the right clear color: vfxManager.cleanUpBuffers(getClearColor()). If you are running into any problems, check out this fork.
• JamesTKhan/Mundus is compatible out of the box, since it uses a copy of NestableFrameBuffer.