Procedural IBL Enhancement in CesiumJS

[ggetz]

Description

This pull request introduces a new system for "procedural" lighting that builds on the Image-Based Lighting (IBL) approach in CesiumJS, aimed at improving the realism and visual quality of the default lighting environment. The changes focus on techniques that create a plausible environment map dynamically based on the current lighting conditions an existing atmospheric scattering code which renders the sky.

Image-Based Lighting Sandcastle

After

Before

Mirror Ball

After

Before

High Altitude Mirror Ball

After

Before

Dynamic Lighting

TODO: More screenshot comparisons

Key Changes

• DynamicEnvironmentMapManager: This new component generates lighting value dynamically based on the existing atmospheric calculations. It uses compute command to generate the following when position or sun position has significantly changed.
  • Environment Map: generated to match lighting conditions based on a central location and the current sun position.
  • Specular Environment Maps: Implements several levels of specular maps, corresponding to various degrees of surface roughness. This allows for more nuanced reflections and highlights that better respond to scene lighting.
  • Spherical Harmonic Diffuse Lighting: Computes of spherical harmonic values for diffuse lighting, enhancing the quality and realism of the light diffusion across different surfaces.
• The computed values from the new IBL setup are directly used in conjunction with the scene's directional light source, primarily the sun, to establish comprehensive lighting effects across models and 3D Tilesets.
• I've chosen to make the DynamicEnvironmentMapManager options accessible via the Model and Cesium3DTileset only, at least for this PR, mainly due to the performance impact if these values are constantly updated.

Additional Updates

• Fix Spherical Harmonic reconstruction: as per the filament maintainers
• Fix for ImageBasedLighting.imageBasedLightingFactor: This property was broken and went unnoticed.
• Removal of ImageBasedLighting.luminanceAtZenith: This property was removed instead of deprecated because there is no direct equivalent in the new implementation, and is likely not widely used (based on the fact that the above bug went unnoticed).
• Modifications to CubeMap: New utility functions in support of the operations in DynamicEnvironmentMapManager, such as copying a texture to a specific face.

Issue number and link

N/A

Testing plan

1. Pull down this branch and run unit tests locally (I had to tweak rendering tests to account for the new lighting. Please double check me here.)
2. Run the development PBR Lighting sandcastle example and make sure all the sliders/buttons work.
3. Review all Sandcastle examples and ensure new lighting parameters are adequate for all cases.

Author checklist

• Fix race condition where the map sometime does not initialize atmosphere values correctly
• Fix slightly askew environment map orientation
• Confirm shadows are affecting lighting correctly
• Expose DynamicEnvironmentMapManager options for Model entities?
• Any adjustments needed for HDR?
• Performance pass and testing
• Cleanup Sandcastle examples-- Streamline existing ones where possible
• Cleanup pass on shaders
• I have submitted a Contributor License Agreement
• I have added my name to CONTRIBUTORS.md
• I have updated CHANGES.md with a short summary of my change
• I have added or updated unit tests to ensure consistent code coverage
• I have updated the inline documentation, and included code examples where relevant
• I have performed a self-review of my code

[github-actions]

Thank you for the pull request, @ggetz!

✅ We can confirm we have a CLA on file for you.

[ggetz]

This update is not strictly necessary, but I found it helpful for debugging.

[ggetz]

@jjspace Could you please take a review pass on this?

[jjspace]

@ggetz Couple initial comments playing with the new sandcastle.

1. When I have a small model selected and modify the height the camera zooms really far out. I saw the comment about not handling small models well but is there a way to mitigate that when modifying the height slider?
2. As mentioned in standup I feel like models are overly blue. Not sure if it's the result of these changes directly or our general visual changes as a whole. For example, this pot at height 400 almost looks like it's glowing blue, like it's emissive itself instead of just reflecting the light around it.
   
   Same with the wicker ball. I'd expect it to be brighter in the areas that are highlighted but I'm not completely sure I'd expect the highlights to be so bluish in color? Maybe I'm just not imagining what the materials would look like correctly
   
3. When modifying the height value it doesn't seem that the lighting is updated as I thought you said it should.
   Pot loaded at 400 then moved to 80,000ish:
   
   Pot reloaded already at 80,000ish:
   

[jjspace]

Had a handful of really small, mostly optional, code comments in addition to the comments above

I don't know if I feel fully qualified to review the shaders but they look ok to me at a cursory glance

[jjspace]

No change needed but curious about this pattern.

It feels like the responsibility for this check should live in the destroy function itself. I feel like you should be able to just call destroy() regardless and let it handle checking if it's already destroyed and essentially turn into a noop function. Then in this function the logic would just be "We know we want to guarantee this is destroyed so call destroy()" and no wrapper check for if it's already destroyed?

[ggetz]

Hm, something to consider. See destroyObject.js for the implementation of this pattern. I'll leave it up to you whether you want to write up a new issue for discussion.

[jjspace]

It's a minor thing but I opened #12266 to discuss

[ggetz]

Thanks for taking a look @jjspace!

1. When I have a small model selected and modify the height the camera zooms really far out. I saw the comment about not handling small models well but is there a way to mitigate that when modifying the height slider?

Yes, but since this is a developer sandcastle example oriented for testing, I think it would be more trouble than it's worth. If you think it's critical, let me know.

2. As mentioned in standup I feel like models are overly blue. Not sure if it's the result of these changes directly or our general visual changes as a whole. For example, this pot at height 400 almost looks like it's glowing blue, like it's emissive itself instead of just reflecting the light around it.

Fair enough. I added a groundAlbedo property to determine how much light is reflected up by the ground (defaulting it to the average ground albedo value on earth). This means that the ground is reflecting more color into the scene. I think this helps balance out the blue from the sky.

For a point of comparison see the glTF Sample Viewer. You can change out the environment map to one with a blue sky for a similar lighting scenario, such as "Wide Street" or "Helipad Golden Hour":

3. When modifying the height value it doesn't seem that the lighting is updated as I thought you said it should.

Good catch. That should be cleaned up now.

I'm taking a last pass on your other feedback.

[jjspace]

Fair enough. I added a groundAlbedo property to determine how much light is reflected up by the ground (defaulting it to the average ground albedo value on earth). This means that the ground is reflecting more color into the scene. I think this helps balance out the blue from the sky.

I think this helps a ton with the appearance of the light around objects like the pot of coals, it's no longer uncannily blue.

I'm taking a last pass on your other feedback.

I think overall this is looking pretty good now, is there more you wanted to do?

[ggetz]

Great, thanks @jjspace!

I think overall this is looking pretty good now, is there more you wanted to do?

I just need to add a few unit tests to reflect my changes– doing so now.

[ggetz]

@jjspace This should be good to go!

[jjspace]

Sorry for the delay on this @ggetz I think this is good to go now. I also tested a handful of other sandcastles that are lighting adjacent and they all seemed to still behave and look as expected (or better)