Switches lm/m^2 to W/m^2 and lm/r^2 to W/r^2 in KHR_lights_punctual.
#2214
Conversation
Lumens are *not* actually a physical unit. They are a perceptual unit, and this leads to all sorts of issues. - Lumens makes RGB difficult to process and reason about, because the same physical power of blue and red produces fewer perceptual lumens than does green. - Lumens produce very big numbers, in the hundreds for a household lightbulb. This may be petty, but is a compatibility issue when viewers and their importers expect numbers around the 0.0-10.0 range. - Lumens (perceptual luminance) are not actually convertible from watts (physical radiance), which are what authoring programs and rendering pipelines work in. This has two consequences: - In logistical terms, it makes implementation of exporters and importers tricky. E.G. KhronosGroup/glTF-Blender-IO#564 was held up for over three years (and is still open) because nobody figured out the conversion function. Other projects, including Three.JS, and the official Khronos Group reference GLTF viewer, seem to also have just ignored the units part of the spec. - In mathematical terms, it means that there is not actually a canonical way to convert from physical watts in authoring and display programs to perceptual lumens for GLTF, or vice versa. They measure different things, so at best you can have a wild guess based on situational equivalence given a bunch of arbitrary assumptions. E.G. 594lm, 609lm, 572lm, and 0.2lm are all equally valid conversions for 1W, depending on the exact wavelength and luminous efficiency function you're using. **This means that as written, the spec's behaviour is actually undefined relative to physically based workflows;** It is actually not possible to build a physically accurate pipeline using lumens as specified in `KHR_lights_punctual`, because the unit itself does not represent a physical quantity. Due to low previous adoption of the lumens unit from the spec (possibly partly due to the aforementioned issues), the disruption from this should hopefully be minimial. I have not found any examples that actually implement or use the lumens units from the spec, including both the Three.JS and official Khronos Group reference GLTF viewers. In any case applications are free to maintain their present, already non-conformant behaviour, and adopt the new behaviour if they wish. See KhronosGroupGH-2213.
Also fix some pre-existing grammar issues.
|
I'm probably not the right person to address your larger comments, but I don't think it can be correct that existing implementations use Watts (with the exception of Blender). See this document, by @bhouston: https://docs.google.com/spreadsheets/d/1Ce9XCC2Ub9eVjAQdYbYdLvlkDBJ_0GtAyRbSF0KJWzE/edit#gid=0 I would certainly love to get export of punctual lights in Blender working, so thanks for investigating here! If I (speaking for three.js) have gotten something wrong here, I think we're happy to make changes to address that. |
|
@donmccurdy So, my question would be: Even if the UI says "lm", what is happening behind the scenes? Maybe it's just because I tend to think more in terms of physics, but I have a hard time picturing a robust rendering pipeline based on a perceptual unit— Do those BDRFs and whatnot ultimately operate in power, which is what watts measure, or in luminance, which is what lumens measure? That said, I agree that if there are already established workflows involving GLTF that depend on lumens/perceptual units, and that is what gets presented by those tools on a UI level and stored on a file level, then this change would not be worth it and should not be made. This PR came from investigation while trying to track down a "root cause" solution in frustration at trying to get Blender→GLTF export making sense according to the reference viewers, and I did try to assess the rest of the ecosystem but documents like that spreadsheet don't really show up on search engines, so it may well not actually make sense outside of that context. That said, if lumens as specified are actually in use, then IMO at least the Khronos Group GLTF viewer should be updated to handle it correctly, but that's another issue. I don't think I've encountered anything explicitly wrong in Three.JS specifically. Different light scaling from Blender, of course, but that's normal. I did notice the docs seemed to say |
|
The established workflows were built around lumens long before glTF's KHR_lights_punctual extension; glTF is following convention more than defining it here. three.js' lighting also predates KHR_lights_punctual. If everyone is broadly using the same units already, perhaps what is missing from the extension is a standardized recommendation on the conversion factor to/from watts? Since, as you point out, it isn't obvious. And then fixing whichever tool(s) have gotten this wrong.
We use candela, or lumens per unit area, for intensity. We also provide an equivalent input called "power", in lumens, but this is just a convenience and is converted when used.
|
|
Other references —
It appears that implementations have just adopted a conversion factor of 683, and are (intentionally) not doing the more complex perceptual version of the conversion. |
|
We are currently relying on the same intensity units that Filament does :-) I would answer like @donmccurdy here:
I am happy to add any new required mode in Babylon if there is a spec change and a simple conversion exist. |
Imho that would be great. Now that you mention it, I realise that that is probably the source of most of my discomfort with trying to get intensity export working in Blender— Having the conversion factor be undefined means that with every export and import operation, the light intensity can change by a significant and arbitrary amount, and though I know glTF isn't meant for interchange, it still uh feels wrong that that can happen. Standardizing the conversion factor would solve that, and the rest of my concerns can be left to the philosophers. Thanks a lot for addressing this in a timely manner, by the way. |
|
On further experiments, let me retract this. I'd made a mistake, and (with the right scene configuration) all three glTF viewers I'm testing (three.js, babylon.js, khronos viewer) are all pretty close to one another. However, using the conversion we've arrived at above, they're all far too bright compared to Blender. So either our conversion is wrong, or we've misunderstood what Blender actually does. I'm worried it may be the latter, and am adding a comment here: https://developer.blender.org/T91035#1431632 I'm doing conversion checks in glTF-Transform, with the script below: import { Light } from '@gltf-transform/extensions';
const scene = document.getRoot().getDefaultScene();
const LUMENS_PER_WATT = 683;
scene.traverse((node) => {
const light = node.getExtension<Light>('KHR_lights_punctual');
if (light) {
// Blender currently exports light intensities without conversion
// from its native radiometric units. See:
// https://github.com/KhronosGroup/glTF-Blender-IO/issues/564
const watts = light.getIntensity();
// Convert radiometric to photometric units, by convention:
// https://google.github.io/filament/Filament.md.html#units
const candela = watts * LUMENS_PER_WATT / (4 * Math.PI);
console.log(`Conversion: ${watts} W → ${candela} cd`);
light.setIntensity(candela);
}
}); |
@donmccurdy My assessment was actually the opposite: That nobody else is actually using the lumens-based units correctly either. E.G. With some GLTF files where I skipped the conversion formula (old plugin), and instead just manually set the intensities to 5, 50, and 500: Text/JSON GLTF so you can open them to verify the intensities. If the viewers were expecting lumens, then you would expect (That is actually why I initially thought the solution should be to just switch the whole spec over to watts— It looked like that was what everyone else was basically already doing. However, as you have explained with that spreadsheet and I have found out in mrdoob/three.js#24788, that is apparently also not the case.) I don't think this has anything to do with Blender behaviour, since we can manually verify that the intensities are appropriate for KHR_lights_punctual (per the spec as written) after the export. |
|
I don't think the problem is on the Blender side as handcrafted/manually verified GLTFs also have the same exposure issues in every viewer I've tried, but some time ago I did write this guide on how to set up Blender to output physically proportional irradiance values as image data: https://blender.stackexchange.com/questions/222959/computing-scene-luminance (The question asked about Cycles, but it should work for EEVEE too— And besides physical parity between the two engines is an explicit(?) goal at the BF, IIRC.) So if it is necessary to confirm that Blender's lighting model behaves internally self-consistently, one could set up a scene where lights of different types shine on different surfaces, and sample and verify that the brightnesses on each surface produced by each type of light are as one would expect given the positioning and nominal wattages of the lights. |
|
@donmccurdy Right. I think I've got it. The renderer.toneMapping = THREE.ReinhardToneMapping;renderer.toneMappingExposure = Math.pow( params.exposure, 5.0 ); // to allow for very bright scenes.If you comment these lines out, then the result also looks very overexposed, similarly to the output from my patch. So the answer then seems to be that all of the model viewers simply aren't configured to display So.... I'm not sure what that means for the purposes of our discussion here. Everything is working as designed, but the design itself has a couple extra steps from what we expected, and it means that default behaviour of viewers for spec-compliant GLTFs is not very nice. I don't have a web stack, so for every test I do I'm basically ....And moreover, is it a problem that with the spec as written and all viewers as designed and implemented, the default behaviour for a standards-compliant GLTF/KHR_lights_punctual file is currently for everything to be extremely overexposed, and for the only way to figure out how to change that to apparently be to find one specific line of code in one specific Three.JS example? That doesn't seem very conducive to efficient workflow and wide adoption. It's certainly caused a lot of confusion here at least, and you'd expect this space to be most ready to figure it out. So... I think that resolves the conversion ratio question. |
|
Your setup to You're correct that three.js, babylon.js, and the Khronos viewer do not include a tone mapping curve by default. The babylon.js and Khronos viewers do have options to enable tone mapping and exposure. For the Khronos viewer, it's under "Advanced": 📸 screenshot
Let's suppose the difference here is really in default tone mapping or exposure. Should we be able to prove that's the cause — and not anything specific to punctual light sources — somehow? Possibly by using an OpenEXR image as a source in both Blender and e.g. three.js? We could also set up export of a linear, open domain [0,∞] image in OpenEXR format from three.js, for any scene, if that's helpful. Or a viewer for such images, with or without tone mapping and exposure. Aside 1 — Something still confuses me about Blender's units. They recommend using a 0.05 W point light to simulate a candela, and the only standard definition of a candle is 1 candela. But by our equation above, 0.05 W * 683 lm/W / (4 * PI m²) = 2.71 cd, not 1 cd. Aside 2 — I've done some work-in-progress implementing Blender-like view transforms in three.js. This isn't part of official three.js today, but you can see some results at https://three-filmic.donmccurdy.com/reference.html. These are implemented as defined in Blender's OCIO config. I'm not sure that's relevant to what we're doing here, but FYI. The images on that page are just using an OpenEXR image as input (not a 3d scene), exported from Blender, then applying the filmic view transform externally in OCIO or three.js. |
If I had before, I don't remember. I never got around to applying that technique much further than what I documented in that StackExchange answer. However, if the goal is simply to show that Blender does in fact use the units it says it uses, then that should be easy enough. Spots and Points define energy the same way, so we can just check that they light up a plane the same: Point and Spot Blender Physicality
(Pay attention to the sampled colour values in the render window screenshots.) And a Sun with intensity of E.G. Sun and Point Blender Physicality
And lastly, to double-check, we can compute physical irradiance to pixel value ratios for all three lamp types to confirm that they all obey the same laws of physics given their nominal units:
(Fun fact: I actually typed out nearly this entire comment yesterday night assuming it would all line up, and then only actually did the experiments and filled in the data this afternoon. Speculative execution like this doesn't always work out for me, but in this case it did (...So far) lol.) (Note: I set up a
If we found a constant factor for the difference in exposure, would that prove it? In KhronosGroup/glTF-Blender-IO#1760, I actually found that simply omitting the conversion factor seemed to visually create equivalence. However, that scene did not include directional lamps. Anyway. Ignoring EXR for now, here's a simple PNG screenshot from Three.JS with linear output, and a render from Blender with linear output, including all three GLTF lamp types. I exported with my KHR_lights_punctual branch, but to keep them on comparable scales set Triple Lamp Blender and Three.JS Linear Views
(Note: I used (Side note: It looks like ortho camera scale is handled differently too... At least that should be fairly straightforwards to harmonize both to the spec 🤷) And here I was going to composite the two together with division, to show that the entire silhouette of Suzanne is flat with a constant ratio. But instead, they ended up so similar that it's easier to compare them based on which is brighter, in which we find that many areas are so similar that they're apparently within the precision limits of the image bit depth: Three.JS Vs. Blender Inequality Comparison
Although for good measure here's also a base Three.JS Vs. Blender Logarithmic Comparison
We could also try to replicate this experiment using an OpenEXR image, I guess, if you think that would help further show it's just the exposure rather an effect of punctual light sources. — But then, what's going to illuminate the EXR? Can't use lamps, since then we're just back here— And are we absolutely sure E.G. that material emissivity and ambient light also work the exact same? I guess we could jack into the pipeline. I personally don't know enough of the API to do that. Barring that, since they turn out exactly the same without the (...And I suppose that brings us back to why this conversation is happening on a PR to switch the spec over to watts— On strictly a physical and visual basis, it now appears that I was actually correct to surmise that existing applications have default exposure settings that are scaled to sanely show Funnily, this is also confirmed by the first ever test I did with KhronosGroup/glTF-Blender-IO#1760. When I first noticed (and expected, based on the conversion factor) that spec-compliant exports would inevitably be grossly overexposed compared to Blender, I added an option to divide all lamp strengths by roughly the same value as the EEVEE Render Vs. Three.JS in lumens with additional divisor to cancel out lumens conversion
I actually gave my opinion on this on
I suspect this is a case of the the guide being written for/by artists more than for developers— "2.71cd" sounds like roughly the level of creative liberty that I would want to take to make sure a physically plausible candle stands out more, and "0.05" sounds like a nice round number I'd settle on to get there.
I don't think that's applicable here, but for me personally it could actually be hugely useful for one of my workflows, so good to know! — Oh, wait. No. With the lamps @donmccurdy This has been a lot of information. Altogether, I think what this means is that we can close this spec PR and then finish the Blender exporter using the conversion factor of |
|
Further finding: Exposure difference factor will be different depending on https://github.com/mrdoob/three.js/search?l=JavaScript&q=physicallyCorrectLights |
This, but also the decay/attenuation curve is different in non-PBR mode. I'd advise testing only in Just curious, are these scenes using the Standard, or Filmic, view transforms in Blender? I don't think it will make a huge difference but I'd be curious to see the fireplace scene with Standard, if it's currently using Filmic.
Exposure controls and tone maps cannot differentiate between sources of light, so light reflected from a surface coming from a point light, or light values plucked directly from an OpenEXR file, are indistinguishable passing into the later parts of the image formation process. The latter should be equivalent to plugging an Image Texture node directly into the Surface socket of a Material Output node in Blender. We can do a similar test in three.js if needed. I don't think we're going to see that the exposure defaults differ by a factor of anything like 683, we will get very similar results. There may be any number of reasons for this.
If you test the model at https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/EmissiveStrengthTest, you'll see that it looks pretty close imported into Blender, three.js, and Babylon.js. A factor of
Just to check – you mean a factor of 1/4π is applied at export?
I'm fine with closing the PR and proceeding with those changes in Blender, since the results appear to be a considerable improvement. I'm still a bit stumped as to why we're seeing what we're seeing. |
Ah, yeah. I just used the live version of the viewer on your website for the screenshots, so I suppose that defaults to
The experimental sample scenes are all using "sRGB" for "Display Device" but "Raw" for "View Transform", which should give linear results I think, then with "Save as Render" enabled when saving images and the colour space set to "Linear" in the compositor (visible in the screenshot) when loading them for the comparison. Where applicable I then also changed "outputEncoding" to "Linear" in the Three.JS viewer to match, which is visible in the screenshot(s). The mouse scene is with "Standard", and also with "sRGB" for "Display Device". All are "Look: None", "Exposure: 0.0", "Gamma: 1.0".
Hm. Well, on a strictly numeric basis, the factor is basically 1.0 (and divided by 4π for points and spots). It's just that for a physically accurate conversion from Blender's watt-based units to lumens for KHR_lights_punctual, we have to multiply light intensities by 683, so that's where the factor comes from. — It's a result of assuming lumens-based units for GLTF but watt-based units for Blender, and thus having to feed Three.JS much larger numbers to start with. The equivalent test for values read from an OpenEXR file in Blender and in Three would also have to include whatever the unit conversion factor there is, I think— We would have to feed a brighter EXR file to Three.JS because those units are also different. So then you would have the exposure "defaults" differ by whatever that factor is. But it's, like, due to how the units are handled (or not handled). (And we can't just treat the file as dimensionless, because then it loses all meaning for representing an amount of light.)
However, Blender currently doesn't convert units on import either. If Blender were to handle imports correctly as per spec, in this case that file would appear several hundred times darker in Blender than in the other viewers. Currently I have an input box allowing users to specify a scaling factor for all light intensities, and a tooltip advising that engines may have different default exposures and suggesting they input "1/600" (Would be "1/683".) there manually for many engines (E.G. if the exported model looks too bright). I personally feel that would have to be the way forwards, assuming nothing else changes. We have to multiply by some factor around 500 to 700, because the spec says that light intensities are in lumens and that is the conversion factor from the units Blender uses. And in that regard, Blender is behaving correctly— A value like 700 lumens is what you can expect from a real-world lightbulb (10W LED, 13W compact fluorescent, or 60W incandescent), and roughly equivalent in practice to 1W of radiated power (as you might define in Blender), which Blender displays correctly/sanely with its default exposures. It's just that when you feed that as a physically accurate, spec-compliant lumens value into every other engine out there, they all display it way too bright with default exposure settings. So barring, E.G. a spec change like this PR that would let us convert instead to a unit that works better with the rest of the ecosystem, the only thing that can be done from Blender is to convert to lumens as specified, and then give users an option to cancel it out/tweak the result if that doesn't play nice with their engine of choice. Note that this also means that all of the example files— If they display "correctly" in the reference viewers— Are all actually ridiculous dark— And that is compensated for by all of the rendering engines having default exposures that are way too bright. E.G. the Again, I think it is worth considering to maybe just change the entire dang spec over to W/m^2 and W/r^2 instead of lm/m^2 and lm/r^2, since that seems to be how the entire ecosystem, all reference viewers, and even the example files are actually behaving, even if they say they're using lumens.
To Point and Spot lamps, yes. That takes them from Blender's total, omnidirectional radiated power to power density per square radian, and brings them into proportion with Sun lamps, which do not require such a factor.
It's because the specs say to use lumens, and the reference implementations say they do use lumens, but in reality all the implementations and even the example files have been set up as if they're using watts. E.G. Even the example file you linked above is full of I'm... Actually going to leave this PR open, and hold off a bit longer on finishing the changes in Blender. If it is possible, I've gone back to thinking that it would be a far, far cleaner and more stable solution in the long term to switch the spec over watt-based behaviour. Three and Babylon already act that way, and so does the Khronos Group viewer— The only real risks for disruption IMO are if (1) somebody downstream of Three/Babylon is using actual lumen values (improbable? I think? Since not even the example files use actual lumen values and without Blender support I'm skeptical whether there even exist any tools that can generate accurate lumen values) or if (2) somebody like Arnold or Dassault or whatever has their own implementation that is scaled to handle lumens correctly that we don't know about. @donmccurdy Summary: Specs say to use lumens/candela/nits. Implementations say they're using lumens. They aren't actually. Even the example files don't make any sense if we take them as lumens-based values. If we implement correct lumens-based exports in Blender, Blender will AFAICT be the only tool in the entire ecosystem that handles the units spec correctly, which.... Could be a problem. I recommending considering changing the spec to watts. That is what everyone is actually already using, so that is the easiest way to bring everything into alignment with the spec without requiring massive downstream changes, and will also allow Blender export defaults to look the same as all other implementations' defaults by simply getting rid of the 683 lumen conversion factor. |
It doesn't feel plausible to me that the whole ecosystem of photometric lighting pipelines is accidentally using Watts, not Lumens, for punctual lights... I don't know of any common renderers, other than Blender, claiming to use Watts? Even to Blender's users, converting their lights to any photometric system seems to be a source of confusion. I am totally willing to believe that the default exposure in three.js might be poor for a typical indoor or daylight scene, though. It's unitless, and we should probably fix that. There's an interesting comment from @romainguy (google/filament#3338 (comment)) saying, in reference to Filament:
I hadn't heard the phrase "unitless" used to describe this situation before, but I think that's what's going on here? We've left exposure at 1, and as a result any comparison of the image on screen to the light intensities is useless. The image is not physically-based. Presumably this hasn't been an issue in general, because few tools export models including lights, and so most viewers have IBL setups with relative intensities that are chosen to match their exposure values. If that's not it, then I've run out of useful ideas to contribute on this thread, and I'm hoping one of the authors of these lighting engines can chime in to correct me. 😇 |
Yes! Actually, I think that's probably a way more accurate way to describe the situation than "accidentally using Watts". I suppose it's not that everyone is accidentally using the "wrong" units, but more that most tools simply do not account for the unit at all, and instead simply map the in-engine RGB values directly to the screen, and I guess nobody's noticed/complained to now because like you say assets and software haven't really made heavy use of lights with physically based strengths yet ....And to me personally that looks like "accidentally using watts", because I'm used to Blender and when Blender decided to move past their own unitless setup they just slapped a "Watts" label on everything (and that happened to work out with mostly sane exposures), so when I see the unitless results I associate it with Watts and and I call it "Watts", but in reality what's happening is that the pipelines are unitless, not that they're using accidentally the wrong unit. Thanks for pointing that out! Concluding the entire ecosystem was blithely using the wrong unit did feel a bit silly, but I wasn't sure how else to describe it. Also, good to see that Filament is exposing renders based on lumens as specified, I think? If I can get that to run I can validate export results against it.
That would also be great. Well, I was reluctant to suggest it as the sheer scale of Three has been a bit overwhelming to me whenever I've tried to fully dive into it, and I assume similar changes would also eventually have to be made to Babylon and possibly others, so.... Somehow trying to change the spec felt like the less disruptive option. 😅 @donmccurdy So.... I think that can settle it? As long as there's also an easy way for viewers like Three to expose it correctly, there's no issue with aligning Blender to the spec with |
Agreed! I'm happy to call this thread 'solved' and go ahead with the your changes in KhronosGroup/glTF-Blender-IO#1760, including the 683 lumens per watt factor. Getting auto exposure in viewers is probably not so easy, but seems worth trying to figure out. There's some good discussion in #2128 about what would be required for glTF exports to provide that information. In the meantime I'll at least get the "exposure" slider on my viewer fixed (donmccurdy/three-gltf-viewer#269), and look into auto-exposure implementations. Thanks for working through all this with me, I've learned a few things here. 🙂 |
- Set `WATTS_TO_LUMENS` to 683 convention. - Add compatibility modes for unitless pipelines and raw Blender units. - Add explanation text. (Currently placeholder, may remove.) - Renamed "Geometry" export subpanel to "Data".
Lumens are not actually a physical unit. They are a perceptual unit, and this leads to all sorts of issues.
KHR_lights_punctual, because the unit itself does not represent a physical quantity.Due to low previous adoption of the lumens unit from the spec (possibly partly due to the aforementioned issues), the disruption from this should hopefully be minimial. I have not found any examples that actually implement or use the lumens units from the spec, including both the Three.JS and official Khronos Group reference GLTF viewers. In any case applications are free to maintain their present, already non-conformant behaviour, and adopt the new behaviour if they wish.
See GH-2213.