how to rescale axis lengths to work around downstream bugs?

[BenWibking]

What do you want to achieve, please describe.

I want to be able to use Paraview to examine openPMD datasets that have very large axis lengths, say 1e20.

However, due to a bug, Paraview cannot render such boxes. See https://gitlab.kitware.com/paraview/paraview/-/issues/21371.

The last resort seems to be i) to rescale the data and write a new file, or ii) rescale the axis lengths dynamically when reading. Is there a straightforward way to do either of these, or do I need to rescale the axis lengths in the simulation code when writing?

Software Environment:
Have you already installed openPMD-api?
If so, please tell us which version of openPMD-api your question is about:

• version of openPMD-api: 0.15.2
• installed openPMD-api via: conda-forge

[BenWibking]

Can openpmd-pipe do this?

[franzpoeschel]

You can try manipulating gridUnitSI, gridSpacing and gridGlobalOffset, see their descriptions here.
If you are using HDF5 files, you can manipulate these items in read-write mode. ADIOS2 does not have that, so one way would be to manipulate the Python scripts of Paraview where it loads these attributes (the openPMD plugin of Paraview is written in Python). Alternatively, you can use openpmd-pipe, yes, and modify the code of that to write the changed attribute values.

[BenWibking]

Is there a github repo for the openPMD plugin for Paraview?

It looks like the plugin is in the Paraview repository (https://gitlab.kitware.com/paraview/paraview/-/blame/master/Wrapping/Python/paraview/algorithms/openpmd.py#L414).

These two lines starting at Wrapping/Python/paraview/algorithms/openpmd.py:414 appear to be the only code that needs to be modified:

            np.array(var.grid_spacing) * var.grid_unit_SI,
            np.array(var.grid_global_offset) * var.grid_unit_SI,

[BenWibking]

You can try manipulating gridUnitSI, gridSpacing and gridGlobalOffset, see their descriptions here.

I don't think this provides a solution. Since the openPMD plugin always passes coordinates to Paraview in SI units, rescaling gridSpacing and gridGlobalOffset and gridUnitSI will yield exactly the same coordinates being passed to Paraview internally by the plugin.

It would be possible to work around the Paraview bug by rescaling gridSpacing and gridGlobalOffset without changing the conversion factor to SI units. That would mean the metadata stored in the file is wrong, which is not an acceptable solution.

I am working on a MR for Paraview that allows for an additional parameter in the *.pmd text file that gives a rescaling factor here: https://gitlab.kitware.com/paraview/paraview/-/merge_requests/6646

[BenWibking]

ADIOS2 does not have that, so one way would be to manipulate the Python scripts of Paraview where it loads these attributes (the openPMD plugin of Paraview is written in Python).

On macOS, it is not possible to edit the contents of Paraview.app/ due to code-signing restrictions. Working around this bug therefore requires building Paraview from source following these instructions: https://github.com/Kitware/ParaView/blob/master/Documentation/dev/build.md#macos.

Unfortunately, it does not build out of the box since it is incompatible with Python 3.12. I had to manually patch the source to get it to build: fix-mpi4py-cython-python3.12.patch

With these workarounds, it finally works. I hope Kitware will fix Paraview, since this was not ideal.

[BenWibking]

VTK is now fixed so that it builds with Python 3.12. Paraview still needs a submodule bump.

[BenWibking]

@ax3l Can you review this PR for Paraview: https://gitlab.kitware.com/paraview/paraview/-/merge_requests/6646?

[ax3l]

I don't think this provides a solution. Since the openPMD plugin always passes coordinates to Paraview in SI units, rescaling gridSpacing and gridGlobalOffset and gridUnitSI will yield exactly the same coordinates being passed to Paraview internally by the plugin.

You can rescale these constants in a way that your box when multiplied with them has unit size, no?

[ax3l]

I am working on a MR for Paraview that allows for an additional parameter in the *.pmd text file that gives a rescaling factor here: https://gitlab.kitware.com/paraview/paraview/-/merge_requests/6646

I do not want to go this way to pass it in the *.pmd file because it will get out of hand and is not a great user API. I added details here: https://gitlab.kitware.com/paraview/paraview/-/merge_requests/6646#note_1465315

[BenWibking]

I don't think this provides a solution. Since the openPMD plugin always passes coordinates to Paraview in SI units, rescaling gridSpacing and gridGlobalOffset and gridUnitSI will yield exactly the same coordinates being passed to Paraview internally by the plugin.

You can rescale these constants in a way that your box when multiplied with them has unit size, no?

Yes, but then the metadata implies it has SI dimensions of 1 m along each axis, which is wrong.