KHR_materials_specular

[proog128]

An alternative to KHR_materials_specular in #1677 that gives more flexibility, see this comment for some motivation.

It adds two parameters to the metallic-roughness material:

• specular: controls the amount of specular reflection in the material.
• specularColor: changes the F0 color of the specular reflection in non-metallic case (similar to specular-glossiness material)

The extension uses some notation introduced in #1717.

[bhouston]

I just proposed to change the other extension into this one: #1741 (comment) I really like this extension and we should adopt it.

[emackey]

I'm not sure I fully grasp why specular is separate from specularColor. Would it be possible to calculate the specular from the max specularColor? So for example is it possible to replace this:

f0 = 0.04 * specular * specularColor
f90 = specular

with something more like this?

float specular = max(specularColor.r, specularColor.g, specularColor.b)
f0 = 0.04 * specularColor
f90 = specular

[proog128]

Computing specular from specularColor will behave a bit different, it will have less flexibility. For example, assuming specularColor is gray [0.5,0.5,0.5], f0 and f90 both would be [0.5,0.5,0.5]. In the original proposal it is possible to control the intensity at f0 and f90 independently: intensity at f90 is controlled via specular, intensity at f0 via specular and specularColor.

Intuitively, specular behaves similar to clearcoatFactor or sheenFactor: it controls the strength of an effect, in this case the specular reflection effect, without changing the effect itself. It doesn't have to be part of f0 or f90, maybe the following (mathematical identical) description is more intuitive:

dielectric_brdf =
  fresnel_mix(
    diffuse_brdf(baseColor),
    microfacet_brdf(roughness^2) * specular,
    f0 = 0.04 * specularColor)

fresnel_mix(base, layer, ior) = base * (1 - specular * fr(ior)) + layer * specular * fr(ior)
fr(ior) = f0 + (1 - f0) * (1 - cos)^5
f0 = 0.04 * specularColor

The following image shows specular from 0 to 1:

The following image shows specular_color from [0,0,0] to [1,1,1]:

The images show the difference between changing the intensity of specular and specular_color: specular color has no effect at grazing angles. In the second image, grazing angles are always reflective, while in the first image the reflection at grazing angles changes.

Another example: varying intensity of a colored specular reflection, specular_color from [0,0,0] to [1,0,0]. At grazing angles, the color is white. At normal angle, the color is red.

The effect is similar if combined with transmission (#1698):

specular from 0 to 1:

specular color from [0,0,0] to [1,1,1]:

specular color from [0,0,0] to [1,0,0]:

And for refraction (#1726):

specular from 0 to 1:

specular color from [0,0,0] to [1,1,1]:

specular color from [0,0,0] to [1,0,0]:

[MiiBond]

Are we not considering specularColor to affect the IOR of the volume? It doesn't appear to in your refraction example above, @proog128 .

There is a PBR argument for having specular colour that I don't think has been mentioned yet. Autodesk Standard Surface uses a specular tint for dielectrics in the regular fashion but, for conductors, they use this value to tint the reflection at grazing angles. Real-world conductors have complex IOR's that vary by wavelength, causing the colour to shift as the angle of reflection changes. Gulbrandsen proposes a artist-friendly parameterization to control this:
http://jcgt.org/published/0003/04/03/paper.pdf

Perhaps we could do the same when metal = 1. It's not much more complex and makes it clear how this param should be used for metals as well as provide a physically-based reason to include it.

[bhouston]

Here is a quick implementation of edge tint driving conductive/complex Fresnel with a comparsion to dieletric real Fresnel:

https://www.shadertoy.com/view/wlySRt

Here is wave-length dependent ior + extinction complex Fresnel compared to dielectric real Fresnel:

https://www.shadertoy.com/view/ttySzt

I am was not aware of any WebGL implementations of this so I created some.

[rsahlin]

Some clarification on my views on the physicality of F0/ior

• This extension targets the metalicroughness workflow, which I see as the way forward for glTF.
• Metalness/dielectric is controlled by the M value.
• Metals absorbs (and re-emits) light - this is what 'tints' the metal
• Dielectricts have none or very weak absorbtion leading to reflection of all wavelengths
• A change of ior does not mean we should alter the metalness value

My view is that the dielectric F0 value should be a scalar (achromatic) reflecting all wavelengths (ie the color of the lightsource)

[rsahlin]

Perhaps we could do the same when metal = 1. It's not much more complex and makes it clear how this param should be used for metals as well as provide a physically-based reason to include it.

Great suggestion - two thumbs up! :-)

[proog128]

Are we not considering specularColor to affect the IOR of the volume?

No, specularColor does not affect the volume IOR.

• Being a color parameter, mapping to IOR would introduce dispersion (wavelength-dependent refractive index).
• Specular color is texturable, but the volume IOR is uniform.

Gulbrandsen proposes a artist-friendly parameterization to control this: [...]

I liked this parametrization, until I heard about this talk: http://renderwonk.com/publications/mam2019/naty_mam2019.pdf
Basically, it says that you shouldn't do RGB magic on top of the full Fresnel equations (which are parametrized by the complex IOR with eta and kappa), because it is less correct than using Schlick's approximation. In addition, the values for reflectivity and edge tint are not intuitive, because they don't match the visual output. Finally, a different parametrization is proposed based on Schlick which doesn't have these problems.

Besides, I am also wondering about a good default value for edge tint. If the extension is enabled, which value do we have to assume to give the same visual result as if it was disabled? Or in other words: what will be the implicit value of this parameter if the extension is not used (i.e., glTF 2.0 metal-rough without extensions)?

[...] My view is that the dielectric F0 value should be a scalar (achromatic) reflecting all wavelengths (ie the color of the lightsource)

I agree completely with your assessment of metallic vs. dielectrics. However, there are a couple of reasons why artistic control over the specular highlight still makes sense in practice. Certainly not for day-to-day usage, but in some cases it gives the material its final touch.

Textiles like silk, satin or felt and leathers may have colored specular highlights.

• There are a few samples in the RGL material database, like acrylic_felt or aniso_sari_silk_2color. Or organic materials: leaf_maple.
• It is mentioned in the documentation to the TAC7 material scanner from X-Rite: "Specular reflections of dielectric materials typically adpt the color of the light source. [...] Theoretically, only metallic surfaces show significant color in their highlights, but many other material types also benefit from allowing colors highlights in their modelling. Especially for textiles it is usually a great advantage to not restrict their representation to white (achromatic) highlights."
• The first paper about the Disney BRDF already introduced a "specularTint" parameter as a "concession for artistic control that tints incident specular towards the base color. Grazing specular is still achromatic."

Multi-layer materials may have a colored specular highlight, because the glossy base is coated with a colored varnish. Or a (very simplistic) car paint material with a diffuse base layer, a colored specular layer for flakes and an achromatic clearcoat layer.

Besides the visual quality/artistic control (that might not be needed in the context of glTF), the biggest advantage of the parametrization consisting of uniform float ior, float specular and float3 specularColor is its compatibility and scalability.

• Compatibility
  • It is compatible to a wide variety of authoring tools and renderers, like UE4, Unity Metal/Rough, Unity Spec/Gloss, Enterprise PBR, Blender Principled, Standard Surface, Octance Universal, AxF, Substance, most of them can be converted without loss.
  • The seamless integration of the specular-glossiness workflow makes transition to the new model very easy. I think unifying the glTF Metal/Rough and glTF Spec/Gloss materials in one material is a huge advantage, making it possible to deprecate KHR_materials_pbrSpecularGlossiness. Engines need to support only one material in one code path.
  • It works well as target for distilling/baking from node-based material models (MDL distiller).
• Scalability
  • Each parameter gives a separate degree of freedom. Together they make the material very powerful, but usually not all are changed at the same time. This makes it possible to optimize renderers for certain combinations. For example, most materials can already be modeled without ior, specular and specularColor. If more control over F0 is needed and it should be spatially-varying, a 1-channel specularColor texture can be added, like in Unreal Engine. Or the specular parameter can be used as alternative, if grazing angles are too bright (fabrics) or to bake small-scale geometry details/shadows (cavities). Last example, a refractive, volumetric glass object can be modeled with just the uniform IOR value, without the need for specular or specularColor.
  • It also scales in terms of "physical plausibility". Artists start off from a "physically-correct" material. Changing the refractive index, either via uniform ior or 1-channel specularColor will keep this property. But if needed, maybe because the model is not sufficient or artistic control is required, changing specular or 3-channel specularColor is possible as last resort. However, it never violates the fundamental light transport principles, positivity, energy conservation and reciprocity.

[bhouston]

> Gulbrandsen proposes a artist-friendly parameterization to control

this: [...]

I liked this parametrization, until I heard about this talk:

http://renderwonk.com/publications/mam2019/naty_mam2019.pdf That is a very interesting presentation. The issue is that if one doesn't go with edgeTint there isn't really agreement in the industry as to what we should use instead. I guess we can go with the proposed solution, h, from this paper but we would be the first I understand. That may mean we are the smartest. :) But I usually feel more comfortable being a follower. What is the name of the "h" parameter? I think it shoud not be edge tint for sure, but maybe we can call it "edge color"?

Besides, I am also wondering about a good default value for edge tint.

There isn't a good default as discussed in the presentation. :/ Autodesk has standardized on edgeTint in their Standard Surface. Maybe we can get some insights from them on its usefulness or downsides? If Autodesk can switch to this it would be nice. Then we have support in Autodesk's suite of tools. Because edge tint can not be converted to edge color and vice versa.

…

On Tue, Mar 10, 2020 at 9:20 AM Tobias Häußler ***@***.***> wrote: Are we not considering specularColor to affect the IOR of the volume? No, specularColor does not affect the volume IOR. - Being a color parameter, mapping to IOR would introduce dispersion (wavelength-dependent refractive index). - Specular color is texturable, but the volume IOR is uniform. Gulbrandsen proposes a artist-friendly parameterization to control this: [...] I liked this parametrization, until I heard about this talk: http://renderwonk.com/publications/mam2019/naty_mam2019.pdf Basically, it says that you shouldn't do RGB magic on top of the full Fresnel equations (which are parametrized by the complex IOR with eta and kappa), because it is less correct than using Schlick's approximation. In addition, the values for reflectivity and edge tint are not intuitive, because they don't match the visual output. Finally, a different parametrization is proposed based on Schlick which doesn't have these problems. Besides, I am also wondering about a good default value for edge tint. If the extension is enabled, which value do we have to assume to give the same visual result as if it was disabled? Or in other words: what will be the implicit value of this parameter if the extension is not used (i.e., glTF 2.0 metal-rough without extensions)? [...] My view is that the dielectric F0 value should be a scalar (achromatic) reflecting all wavelengths (ie the color of the lightsource) I agree completely with your assessment of metallic vs. dielectrics. However, there are a couple of reasons why artistic control over the specular highlight still makes sense in practice. Certainly not for day-to-day usage, but in some cases it gives the material its final touch. Textiles like silk, satin or felt and leathers may have colored specular highlights. [image: felt] <https://user-images.githubusercontent.com/2821679/76315557-32ec6580-62d9-11ea-9153-a76e047e2794.png> [image: webbing] <https://user-images.githubusercontent.com/2821679/76315570-38e24680-62d9-11ea-8908-5b58123cb51c.png> [image: leather] <https://user-images.githubusercontent.com/2821679/76315561-35e75600-62d9-11ea-8bca-4c7975ff4e4a.png> - There are a few samples in the RGL material database <http://rgl.epfl.ch/materials>, like acrylic_felt or aniso_sari_silk_2color. Or organic materials: leaf_maple. - It is mentioned in the documentation to the TAC7 material scanner from X-Rite: "Specular reflections of dielectric materials typically adpt the color of the light source. [...] Theoretically, only metallic surfaces show significant color in their highlights, but many other material types also benefit from allowing colors highlights in their modelling. Especially for textiles it is usually a great advantage to not restrict their representation to white (achromatic) highlights." - The first paper about the Disney BRDF <https://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf> already introduced a "specularTint" parameter as a "concession for artistic control that tints incident specular towards the base color. Grazing specular is still achromatic." Multi-layer materials may have a colored specular highlight, because the glossy base is coated with a colored varnish. Or a (very simplistic) car paint material with a diffuse base layer, a colored specular layer for flakes and an achromatic clearcoat layer. Besides the visual quality/artistic control (that might not be needed in the context of glTF), the biggest advantage of the parametrization consisting of uniform float ior, float specular and float3 specularColor is its compatibility and scalability. - Compatibility - It is compatible to a wide variety of authoring tools and renderers, like UE4, Unity Metal/Rough, Unity Spec/Gloss, Enterprise PBR, Blender Principled, Standard Surface, Octance Universal, AxF, Substance, most of them can be converted without loss. - The seamless integration of the specular-glossiness workflow makes transition to the new model very easy. I think unifying the glTF Metal/Rough and glTF Spec/Gloss materials in one material is a huge advantage, making it possible to deprecate KHR_materials_pbrSpecularGlossiness. Engines need to support only one material in one code path. - It works well as target for distilling/baking from node-based material models (MDL distiller). - Scalability - Each parameter gives a separate degree of freedom. Together they make the material very powerful, but usually not all are changed at the same time. This makes it possible to optimize renderers for certain combinations. For example, most materials can already be modeled without ior, specular and specularColor. If more control over F0 is needed and it should be spatially-varying, a 1-channel specularColor texture can be added, like in Unreal Engine. Or the specular parameter can be used as alternative, if grazing angles are too bright (fabrics) or to bake small-scale geometry details/shadows (cavities). Last example, a refractive, volumetric glass object can be modeled with just the uniform IOR value, without the need for specular or specularColor. - It also scales in terms of "physical plausibility". Artists start off from a "physically-correct" material. Changing the refractive index, either via uniform ior or 1-channel specularColor will keep this property. But if needed, maybe because the model is not sufficient or artistic control is required, changing specular or 3-channel specularColor is possible as last resort. However, it never violates the fundamental light transport principles, positivity, energy conservation and reciprocity. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1719?email_source=notifications&email_token=AAEPV7N6YAO75TCGNDQRYIDRGY5BRA5CNFSM4JUX56NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOLL2LQ#issuecomment-597081390>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEPV7L4BF7A7QYFNB26H4DRGY5BRANCNFSM4JUX56NA> .

-- *Ben Houston*CTO *M: *+1-613-762-4113 bhouston@threekit.com Ottawa, ON [image: threekitlogo1568384278.png] <https://www.threekit.com/> Create a better visual customer experience with Threekit <https://www.threekit.com/> [image: Learn how to scale product visuals with your growing business] <http://signatures.threekit.com/uc/5d0cfd47860b6864aee40938/c_5da8cc74c90a0e0043fe0323/b_5dade3e4e85fae00caea3d76>

[chazbg]

Since we are on this topic here's some info on how V-Ray does the metallic Fresnel:

Currently V-Ray does the conductor Fresnel calculations differently than Ole Gulbrandsen's “Artist friendly metallic Fresnel” paper because of the same concerns as:
https://diglib.eg.org/bitstream/handle/10.2312/mam20191305/007-011.pdf

V-Ray's calculation is actually really simple and it uses the full dielectric Fresnel equation to approximate the metallic Fresnel:

f=getFresnelCoeff(viewDir, normal, refractDir, ior);
metallicFresnel=lerp(base, specular, f);

Where getFresnelCoeff is the full dielectric Fresnel equation (R_eff in https://en.wikipedia.org/wiki/Fresnel_equations).

This approximates metallic Fresnel better than Gulbrandsen's approach.
Also, V-Ray doesn't use Schlick's approximation because it increases the error noticeably compared to the full Fresnel equation.
More information can be found at: https://github.com/vkoylazov/metalness

Here’s a link to a video with some results for various metals so you don’t have to compile and run it yourself:
https://drive.google.com/file/d/130wC2tHtNEi6V1m2eum9pnNAufSKDruK/view?usp=sharing

In the video, the X axis is the incident angle and the Y axis is the Fresnel reflectance. The plots are coded like this:

• Dots: Gulbrandsen's metallic Fresnel for red/green/blue wavelengths
• Dashed lines: The actual complex Fresnel reflectance with long dashed graphs for red/green/blue wavelengths
• Solid lines: The metallic Fresnel derived from the dielectric Fresnel, as used in V-Ray

[bhouston]

Currently V-Ray does the conductor Fresnel calculations differently than Ole Gulbrandsen's “Artist friendly metallic Fresnel” paper because of the same concerns as: https://diglib.eg.org/bitstream/handle/10.2312/mam20191305/007-011.pdf

So V-Ray's implementation agrees with what Tobias is proposing and what I like -- this is just the paper form of the earlier presentation. V-Ray use of this means we wouldn't be the first but following in the footsteps of a great renderer.

Also, V-Ray doesn't use Schlick's approximation because it increases the error noticeably compared to the full Fresnel equation. More information can be found at: https://github.com/vkoylazov/metalness

This does differ from Tobias's suggested approach. The full Fresnel is not that much more expensive than Schlick in my opinion - 3 to 4x more computational intense but thankfully no texture fetches. I think a ShaderToy comparison of all this would be useful so we can see the perceptual differences first hand on common use cases.

…

On Tue, Mar 10, 2020 at 4:11 PM chazbg ***@***.***> wrote: Since we are on this topic here's some info on how V-Ray does the metallic Fresnel: Currently V-Ray does the conductor Fresnel calculations differently than Ole Gulbrandsen's “Artist friendly metallic Fresnel” paper because of the same concerns as: https://diglib.eg.org/bitstream/handle/10.2312/mam20191305/007-011.pdf V-Ray's calculation is actually really simple and it uses the full dielectric Fresnel equation to approximate the metallic Fresnel: f=getFresnelCoeff(viewDir, normal, refractDir, ior); metallicFresnel=lerp(base, specular, f); Where getFresnelCoeff is the full dielectric Fresnel equation (R_eff in https://en.wikipedia.org/wiki/Fresnel_equations). This approximates metallic Fresnel better than Gulbrandsen's approach. Also, V-Ray doesn't use Schlick's approximation because it increases the error noticeably compared to the full Fresnel equation. More information can be found at: https://github.com/vkoylazov/metalness Here’s a link to a video with some results for various metals so you don’t have to compile and run it yourself: https://drive.google.com/file/d/130wC2tHtNEi6V1m2eum9pnNAufSKDruK/view?usp=sharing In the video, the X axis is the incident angle and the Y axis is the Fresnel reflectance. The plots are coded like this: Dots: Gulbrandsen's metallic Fresnel for red/green/blue wavelengths Dashed lines: The actual complex Fresnel reflectance with long dashed graphs for red/green/blue wavelengths Solid lines: The metallic Fresnel derived from the dielectric Fresnel, as used in V-Ray — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

-- Ben Houston CTO M: +1-613-762-4113 bhouston@threekit.com Ottawa, ON Create a better visual customer experience with Threekit

[rsahlin]

I agree completely with your assessment of metallic vs. dielectrics. However, there are a couple of reasons why artistic control over the specular highlight still makes sense in practice. Certainly not for day-to-day usage, but in some cases it gives the material its final touch.

@proog128 My opinion is that the examples you provide (textiles and multi layer materials) are part of new parameters that we have not yet modelled.

• It seems the textile examples show effects of absorption and/or scattering - We should parameterize that and not introduce a 'fudge value' for a short term solution.
• Multi layer materials are not yet part of our model - We should find a solution for this in the future - not add a short term fudge.
• Fortnite type of 'toon' rendering is not part of PBR and should be put in some other extension if needed.

A lot of it boils down to what Emmet said - we need to anchor our values in a physical (photorealistic) model so that we can compare with a photo and determine what the correct result should look like.

Please keep in mind that, for the moment at least, we are discussing dielectric F0 and not conductive (metal) tinting.

[proog128]

A lot of it boils down to what Emmet said - we need to anchor our values in a physical (photorealistic) model so that we can compare with a photo and determine what the correct result should look like.

There is a difference between a physically-based BRDF (which is positive, reciprocal and energy-conserving) and a certain mathematical function that approximates photo-realistic materials (GGX microfacet model, metallic/roughness model, etc.).

A physically-based BRDF is prerequisite for consistency across renderers, because it makes the material independent of the rendering algorithm. Everything proposed so far (including Spec/Gloss) can be implemented in a physically-based BRDF, see the equations in Enterprise PBR.

The model is needed because a tabulated (4D) BRDF contains thousands of data points, requiring lots of space. We want a simple analytical function that models (photo-)realistic materials.

I don't think we will ever be able to build a BRDF model that simulates all effects needed for photo-realistic materials. Especially not if it should still be possible to run it in real-time. There is research in this area, but this is out of scope for glTF. Note that a node-based material system like MDL or OSL is also very limited compared to "real-world physics", it also contains lots of phenomenological motivated (artistic) freedom.

[rsahlin]

Good point @proog128 - you are of course correct.
At the same time the line between the two can be very thin at times - which I think is the case for this dielectric F0.

In this case I guess I am in the group leaning towards waiting with the articstic freedom parameter.
:-)

[rsahlin]

I just had a discussion with the material expert at Ikea.
He pointed at another problem with the artistic freedom parameter - it will lead to inconsistencies in materials.
Simply because there are a couple of parameters to use without clear connection to desired output - roughness, occlusion and dielectric F0 rgb.
Designers will use these in different ways when creating, for instance textile, materials - leading to differences when rendered using glTF.

I would love if we could have as a goal to do a textile extension in a near future and not do RGB in this extension.
We currently don't see a big need for multi-layer support.

[proog128]

Simply because there are a couple of parameters to use without clear connection to desired output - roughness, occlusion and dielectric F0 rgb.
Designers will use these in different ways when creating, for instance textile, materials - leading to differences when rendered using glTF.

Could you please explain your concerns a bit more? So far the extension describes the output in terms of a BSDF. If you put the formula for the BSDF into code, you will get the exact same visual output.

Sooner or later I'd like to describe it in a more abstract way, like in a node-based shading model, to give implementations a bit more freedom. I started doing it in #1717 and #1738, see this (slightly outdated) image:

If you need more math, have a look at the Enterprise PBR spec. There is also a set of reference renderings to compare different implementations. The approach of comparing renderers via images is not new, as far as I know it is also the used successfully in the Fidelity Tests of Filament/<model-viewer> and in AxF (I shared the white-paper about this format on the 3D Formats Mailing List). So whatever designers do with these parameters, the rendering result will be consistent if implementations adhere to the standard.

I would love if we could have as a goal to do a textile extension in a near future and not do RGB in this extension.

I don't think it's an either-or. There are so many different types of fabrics/textiles/cloth/felt/hair that it will be hard to make a single extension for it. There is KHR_materials_sheen (#1688) which is useful for certain effects commonly found in textiles. Do you have a specific model in mind?

[proog128]

In the first case, are these valid conversions?

There is indeed one point for which I am unsure what we should recommend. The ior can be either 0 or a high number (1000): f0 = ((1-ior)/(1+ior))^2 becomes 1 for ior=0 and for ior=1000 (almost). I just noticed that we now require a minimum IOR of 1in KHR_materials_ior. So we might get better comaptibility if we change the recommendation to 1000 or another high number.

[donmccurdy]

There was a bug in my code. 😇 Babylon works fine with ior=0 or ior=1000, although it's a good point that we should recommend something that KHR_materials_ior allows. Results now look very close:

spec/gloss	metal/rough + specular

[emackey]

I love that this works so well.

My only very slight hesitation comes from an XKCD cartoon...

• Situation: There are two competing standards, metal/rough vs. spec/gloss
• Me: I know! Let's unify on metal/rough by adding specular as a parameter!
• Situation: There are now three competing standards.

Well not separate standards, but three ways of achieving the same effect within a glTF PBR material. Looks like we'll soon have a new sample model showing the 3 ways of texturing that water bottle (metal/rough vs spec/gloss vs spec/nonmetal/rough).

[donmccurdy]

I wouldn't really consider metal/rough and nonmetal/spec/rough to be competing workflows, the former is clearly preferable for artists, who aren't likely to find ior=Infinity very intuitive. It's mainly useful as a reliable conversion from the spec/gloss workflow — according to the Substance devs, "there is no safe way to automate" a conversion from spec/gloss to metal/rough otherwise.

I don't think we need to feature prominent examples of the ior=Infinity case in the sample models repository — it's a useful workaround, not a best practice.

This comment has been addressed.

[emackey]

Merging this. Status will remain "Draft" for at least a few more weeks, but it will be hosted in the main glTF repository.