EXT_materials_bump #2339

bhouston · 2023-10-22T18:21:31Z

This is a custom extension that we are using. Figured it may be of interest to others.

Implemented for ThreeJS here: mrdoob/three.js#27036

Benefits of bump maps are primarily these two:

Bump maps are much more compact that normal maps and lead to smaller glTF files as well as less GPU memory requirements. This is because the normal map has to store a full 3D vector per texel, while the bump map only needs to store a single scalar value per texel. Thus bump maps require 3 times less data than normal maps for transmission and GPU storage. Normal maps are also very sensitive to texture compressions methods, while bump maps are less sensitive.
Bump maps are useful for capturing small details in combination with a normal map for large scale structures. For example, when rendering human faces, there are usually two layers of surface pertubation detail, the unwrapped normal map is used for representing wrinkles, and then a repeating bump map is used for skin pore details. This is also the case for many other materials such as wood, metal, and stone, where there is both macro and micro details.

I can also add code to derive a normal map from a bump map and vice versa to the extension text. Normal and bump maps are interchangeable for the most part.

bhouston · 2023-10-23T19:48:48Z

Example glTF that uses this extension exported via ThreeJS's glTF exporter is here: https://drive.google.com/file/d/1NX_LGF0Yp_F5ceWhStcoYbH_UGpuhFqz/view?usp=share_link. You can also import it into the ThreeJS editor via the ThreeJS glTF importer. I can contribute this example to glTF-SampleAssets if desired.

extensions/2.0/Vendor/EXT_materials_bump/README.md

bhouston · 2023-10-23T20:39:11Z

@donmccurdy would it make you feel better if I rename this to be EXT_materials_displacement? And then I can add in text that says this can be rendered either as normal perturbation method or as a surface displacement? I think the main difference is that if I call it EXT_materials_displacement, the displacementScale factor can go above 1. Let me know...

extensions/2.0/Vendor/EXT_materials_bump/README.md

…maps.

donmccurdy · 2023-10-24T02:09:18Z

My main concern would be (potentially) causing bump maps to become much more common in the ecosystem. There is currently no other way to achieve detail maps in glTF, so tools would have to implement an outdated feature (bump maps) in order to support a very useful modern feature (detail maps). Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

Perhaps I'm overthinking the risk. I don't need to block this if others are happy with the proposal! I see displacement and parallax maps more positively.

Aside: I'd be very interested in the MaterialX story for vertex displacement and animation. Does that mean MaterialX input to a displacementTexture socket, or is it a separate thing? But maybe off topic here. 😅

bhouston · 2023-10-24T02:20:54Z

@donmccurdy wrote:

Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

You do realize that right now Sketchfab just discards the bump map right if it exists? They also support, like ThreeJS both a normal and a bump. So we currently just have information loss. This addresses that.

bhouston · 2023-10-24T02:23:50Z

My main concern would be (potentially) causing bump maps to become much more common in the ecosystem.

It is possible to convert on load to a normal map from a bump map. So you can stick with normal mapping through out your engine if you so desired.

I'd be very interested in the MaterialX story for vertex displacement and animation.

That is great too!

I guess I do not see this as a step backwards, but rather just supporting what is already in the ecosystem.

bhouston · 2023-10-24T02:36:53Z

so tools would have to implement an outdated feature (bump maps) in order to support a very useful modern feature (detail maps)

BTW MaterialX supports bump maps: https://materialx.org/docs/api/class_height_to_normal_node_glsl.html. Pixar Renderman also has bump map support: https://rmanwiki.pixar.com/display/REN24/PxrBumpMixer (which was all reworked based on new 2020-era research outlined here: https://jcgt.org/published/0009/03/04/). V-Ray and Arnold and Blender also support bump maps.

echadwick-artist · 2023-10-24T12:33:33Z

As far as packing goes, I compiled a list of which channels are still unused by extensions which pack textures. However there's the downside of potentially incurring more compression noise from having disparate data in neighboring channels. /shrug

KHR_materials_anisotropy

RGB = anisotropyTexture
Alpha = unused

KHR_materials_clearcoat

RGB = clearcoatNormalTexture
Red = clearcoatTexture
Green = clearcoatRoughnessTexture
Blue = unused
Alpha = Unused

KHR_materials_iridescence

Red = iridescenceTexture
Green = iridescenceThicknessTexture
Blue = unused
Alpha = Unused

KHR_materials_sheen

RGB = sheenColorTexture
Alpha = sheenRoughnessTexture

KHR_materials_specular

RGB = specularColorTexture
Alpha = specularTexture

KHR_materials_transmission

Red = transmissionTexture

KHR_materials_volume

Green = thicknessTexture
Blue = unused
Alpha = Unused

echadwick-artist · 2023-10-24T15:27:51Z

As far as packing goes, I compiled a list of which channels are still unused by extensions which pack textures.

If the plan is to pack the bump into a shared texture, I would prefer we use the Blue channel, since there are three textures where it can fit.

lexaknyazev · 2023-10-24T15:36:25Z

If the plan is to pack the bump into a shared texture, I would prefer we use the Blue channel, since there are three textures where it can fit.

Adding an unrelated blue channel to already uncorrelated two-component textures would greatly increase compression artifacts, in some cases preventing use of compressed data altogether.

bhouston · 2023-10-26T14:35:53Z

@lexaknyazev Threekit now supports this extension and mrdoob over at ThreeJS is also supportive - will merge my PR to ThreeJS once it is merged here: mrdoob/three.js#27036 (comment). This means that we have two implementations.

How do we get this merged into Khronos as a community standard then?

I can also commit to adding it to the @donmccurdy gltf-transform tool as an optional extension there as well if he lets me.

bhouston · 2023-10-26T14:36:52Z

Adding an unrelated blue channel to already uncorrelated two-component textures would greatly increase compression artifacts, in some cases preventing use of compressed data altogether.

I suggest we just use the red channel and do not merge it with any other textures for the time being. I think it doesn't fit well with any of the ones @echadwick-artist mentions - it isn't a natural fit with those extensions anyhow.

donmccurdy · 2023-10-26T14:57:24Z

Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

You do realize that right now Sketchfab just discards the bump map right if it exists? They also support, like ThreeJS both a normal and a bump. So we currently just have information loss.

I did not; that is unfortunate. 😭

...would it make you feel better if I rename this to be EXT_materials_displacement? And then I can add in text that says this can be rendered either as normal perturbation method or as a surface displacement?

I think it's probably best to keep these as two separate techniques.

If only three.js and ThreeKit are interested in supporting the extension, it should probably use a THREE_* prefix. If more then EXT_* makes sense.

I don't feel strongly enough to say more than I have, and will bow out here. It would be helpful if other vendors could give some rough sense of positive/neutral/negative signal — however you feel is best for your applications and communities. 🙂

bhouston · 2023-10-26T15:26:43Z

BTW Alban over at Sketchfab liked the extension when I mentioned it to him - finally all those models over at Sketchfab that have bumps can actually export that info: https://twitter.com/benhouston3d/status/1716640855455412305

bhouston · 2023-11-24T03:40:55Z

This is now officially supported in ThreeJS, was merged today. Thus it is supported in 2 separate systems - ThreeJS and Threekit. :)

lexaknyazev · 2023-12-11T16:48:29Z

extensions/2.0/Vendor/EXT_materials_bump/schema/material.EXT_materials_bump.schema.json

+            "description": "The strength of the bump displacement.",
+            "default": 1.0,
+            "minimum": 0.0,
+            "maximum": 100.0,


The maximum limit seems arbitrary. Could it be removed?

I'd suggest removing the minimum as well to allow for negative scales, like the normalTexture.

lexaknyazev · 2023-12-20T13:47:35Z

extensions/2.0/Vendor/EXT_materials_bump/README.md

+
+## Overview
+
+This extension adds one parameters to the metallic-roughness material: `bump`.


Typo: one parameter

lexaknyazev · 2023-12-20T13:48:31Z

extensions/2.0/Vendor/EXT_materials_bump/README.md

+
+This extension adds one parameters to the metallic-roughness material: `bump`.
+
+`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel.  The height is then used to perturb the normal direction.


Please change "grayscale" to "single-channel".

lexaknyazev · 2023-12-20T13:50:39Z

extensions/2.0/Vendor/EXT_materials_bump/README.md

+
+`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel.  The height is then used to perturb the normal direction.
+
+Bump maps were introduced into computer graphics in 1978 by Blinn in "Simulation of Wrinkled Surfaces, while normal maps were introduced in 1989 by Cohen et al, "Appearance-Preserving Simplification."


Please split the rest of this section (starting from this line) into a new section called "Motivation" with the following text prepended:

This section is non-normative.

lexaknyazev · 2023-12-20T13:51:26Z

extensions/2.0/Vendor/EXT_materials_bump/README.md

+| **bumpFactor** | `number` | The strength of the specular reflection. | No, default: `1.0`|
+| **bumpTexture** | [`textureInfo`](https://www.khronos.org/registry/glTF/specs/2.0/glTF-2.0.html#reference-textureinfo) | A texture that defines the strength of the specular reflection, stored in the red (`R`) channel. This will be multiplied by bumpFactor. | No |


Descriptions are clearly from a different extension.

lexaknyazev · 2023-12-20T13:54:05Z

extensions/2.0/Vendor/EXT_materials_bump/schema/material.EXT_materials_bump.schema.json

+        "bumpTexture": {
+            "allOf": [ { "$ref": "textureInfo.schema.json" } ],
+            "description": "A texture that defines the displacement in the red channel.",
+            "gltf_detailedDescription": "A texture that defines the displacement in the red channel. This will be multiplied by bumpFactor."


"bumpFactor" -> "the bump factor"

lexaknyazev · 2023-12-20T13:55:19Z

extensions/2.0/Vendor/EXT_materials_bump/schema/material.EXT_materials_bump.schema.json

+            "default": 1.0,
+            "minimum": 0.0,
+            "maximum": 100.0,
+            "gltf_detailedDescription": "This parameter scales the amount of normal perturbation by the bump map on the surface normal."


Please prepend the detailed description with the short description from L10.

bhouston added 2 commits October 22, 2023 14:20

first draft of bump extension.

eecc9f0

add use cases, TODOs

da11125

bhouston mentioned this pull request Oct 23, 2023

GLTFLoader+GLTFExporter: Initial support for 'EXT_materials_bump' mrdoob/three.js#27036

Merged

donmccurdy reviewed Oct 23, 2023

View reviewed changes

lexaknyazev reviewed Oct 23, 2023

View reviewed changes

extensions/2.0/Vendor/EXT_materials_bump/README.md Outdated Show resolved Hide resolved

bhouston added 2 commits October 23, 2023 16:41

remove draft

219e90d

mention that both ThreeJS and BabylonJS support both bump and normal …

aa25bd1

…maps.

bhouston added 2 commits October 24, 2023 09:53

adad mention the limitation of bump maps and references.

23153c4

fix left over KHR_ prefixes with EXT_ prefixes.

2d5807a

be sure that we are using the red channel for bump texture.

2a08ad4

lexaknyazev self-requested a review October 26, 2023 15:41

lexaknyazev requested changes Dec 20, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EXT_materials_bump #2339

EXT_materials_bump #2339

bhouston commented Oct 22, 2023 •

edited

Loading

bhouston commented Oct 23, 2023

bhouston commented Oct 23, 2023

donmccurdy commented Oct 24, 2023

bhouston commented Oct 24, 2023

bhouston commented Oct 24, 2023

bhouston commented Oct 24, 2023 •

edited

Loading

echadwick-artist commented Oct 24, 2023 •

edited

Loading

echadwick-artist commented Oct 24, 2023

lexaknyazev commented Oct 24, 2023

bhouston commented Oct 26, 2023 •

edited

Loading

bhouston commented Oct 26, 2023

donmccurdy commented Oct 26, 2023 •

edited

Loading

bhouston commented Oct 26, 2023 •

edited

Loading

bhouston commented Nov 24, 2023

lexaknyazev Dec 11, 2023

DanielSturk Jan 18, 2024

lexaknyazev Dec 20, 2023

lexaknyazev Dec 20, 2023

lexaknyazev Dec 20, 2023

lexaknyazev Dec 20, 2023

lexaknyazev Dec 20, 2023

lexaknyazev Dec 20, 2023


		## Overview

		This extension adds one parameters to the metallic-roughness material: `bump`.


		This extension adds one parameters to the metallic-roughness material: `bump`.

		`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel. The height is then used to perturb the normal direction.


		`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel. The height is then used to perturb the normal direction.

		Bump maps were introduced into computer graphics in 1978 by Blinn in "Simulation of Wrinkled Surfaces, while normal maps were introduced in 1989 by Cohen et al, "Appearance-Preserving Simplification."

		\| bumpFactor \| `number` \| The strength of the specular reflection. \| No, default: `1.0`\|
		\| bumpTexture \| [`textureInfo`](https://www.khronos.org/registry/glTF/specs/2.0/glTF-2.0.html#reference-textureinfo) \| A texture that defines the strength of the specular reflection, stored in the red (`R`) channel. This will be multiplied by bumpFactor. \| No \|

EXT_materials_bump #2339

Are you sure you want to change the base?

EXT_materials_bump #2339

Conversation

bhouston commented Oct 22, 2023 • edited Loading

bhouston commented Oct 23, 2023

bhouston commented Oct 23, 2023

donmccurdy commented Oct 24, 2023

bhouston commented Oct 24, 2023

bhouston commented Oct 24, 2023

bhouston commented Oct 24, 2023 • edited Loading

echadwick-artist commented Oct 24, 2023 • edited Loading

KHR_materials_anisotropy

KHR_materials_clearcoat

KHR_materials_iridescence

KHR_materials_sheen

KHR_materials_specular

KHR_materials_transmission

KHR_materials_volume

echadwick-artist commented Oct 24, 2023

lexaknyazev commented Oct 24, 2023

bhouston commented Oct 26, 2023 • edited Loading

bhouston commented Oct 26, 2023

donmccurdy commented Oct 26, 2023 • edited Loading

bhouston commented Oct 26, 2023 • edited Loading

bhouston commented Nov 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhouston commented Oct 22, 2023 •

edited

Loading

bhouston commented Oct 24, 2023 •

edited

Loading

echadwick-artist commented Oct 24, 2023 •

edited

Loading

bhouston commented Oct 26, 2023 •

edited

Loading

donmccurdy commented Oct 26, 2023 •

edited

Loading

bhouston commented Oct 26, 2023 •

edited

Loading