Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dequantize WEB3D_quantized_attributes #154

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

javagl
Copy link
Contributor

@javagl javagl commented Nov 2, 2024

When the upgrade command encounters tile content (like B3DM) that still contains glTF 1.0 data, then this is first converted into glTF 2.0 using gltf-pipeline. Any other update steps are then performed based on this glTF 2.0 data.

When the glTF 1.0 content used the WEB3D_quantized_attributes extension, then gltf-pipeline will not handle this: It will upgrade most of the glTF to 2.0, but does not know what to do with this extension. It therefore leaves the extension objects in the accessors, and keeps the extension in the extensionsRequired declaration. Subsequent upgrade steps (mostly based on glTF-Transform) could then not be performed, because glTF-Transform bailed out due to the unsupported (but required) extension.

There could be different ways of addressing this.

  • One could consider adding support for this extension to glTF-Transform, but ... meh...: It is an extension for glTF 1.0, and adding support for that in glTF-Transform (which only operates on glTF 2.0) would be odd...
  • Theoretically, it would make most sense to add this as part of the glTF 1.0 -> glTF 2.0 upgrade in gltf-pipeline. But dequantizing and replacing an accessor requires a bit of manual fiddling (adding accessors and buffer views, removing old accessors and buffer views...)
  • The approach that is taken here: The upgrade step will check whether the WEB3D_quantized_attributes extension is used. If it is used, then the decodeMatrix arrays from the extension objects of the accessor are extracted, and the extension declaration is removed. The resulting document is read with glTF-Transform, and the accessors that contained the extension object are replaced with dequantized ones.

This is a bit "sneaky": At some point, the whole thing is a glTF 2.0 document that still contains the quantized accessors, and they are decoded "manually" based on the knowledge from its original representation. The reason for this approach is... mainly that juggling around with the accessors is so convenient in the glTF-Transform document...

There are currently zero tests and examples. It's hard to find B3DM data that still contains glTF 1.0, and even more to find such data that additionally contains WEB3D_quantized_attributes (this wasn't really widely used). So this is just a draft PR, considered to be a "preview" or "workaround". Feedback is still appreciated.

@javagl
Copy link
Contributor Author

javagl commented Nov 3, 2024

In some way, it could be cool if gltf-pipeline could just translate WEB3D_quantized_attributes into KHR_mesh_quantization. It would only consist of tweaking around in the JSON data itself. The decoding of the binary values could then be left to the loader, which has to handle the resulting KHR_mesh_quantization. But unfortunately, WEB3D_quantized_attributes is in some way more "powerful" than KHR_mesh_quantization: It would be trivial to translate KHR_mesh_quantization into WEB3D_quantized_attributes. But there can be WEB3D_quantized_attributes that can not be represented as KHR_mesh_quantization.

@javagl
Copy link
Contributor Author

javagl commented Dec 19, 2024

There recently has been a bit more demand for upgrading data that contains glTF with the WEB3D_quantized_attributes extension. So I think that it could make sense to make this part of the default upgrade command. When the input does not contain the extension, this change will not have any effects. And when it does contain the extension, then this is a "best-effort" approach of trying to salvage that data.

This branch has already been tested with a real-world data set, and seemed to work in that case. I now have added example data with the irritatingly specific name tilesetWithB3dmWithGltf1WithWeb3dQuantizedAttributes. This contains a single triangle with its positions quantized to unsigned shorts. (I had to manually fiddle this together in JglTF, because there are very few tools that still can create glTF 1.0 to begin with, and adding that (old, unsupported) extension wasn't possible out-of-the-box...)

The data could be loaded in CesiumJS, but not in other runtimes, because they cannot handle glTF 1.0 and this extension. Applying the upgrade --targetVersion 1.1 command to this generates a tileset with a GLB that passes validation and contains the dequantized data.

Note: This data is currently not part of any specs. It is only processed as part of the TilesetUpgraderDemo.ts. These upgradeTileset example data sets should probably be added to the specs at some point, using the "golden output"-approach that is already used for other tests.

@javagl javagl marked this pull request as ready for review December 19, 2024 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant