Mujoco interactive viewer

[tsgtdss583]

Hi,

I'm a beginner in MUjoco, and i am trying to use the python-binding to run some model in real-time while changing some motion parameteres. I went through the tutorial . however, they used a mediapy package to replay the scene after the simulation is completed. Then I recently went to the interactive viewer but I got an import error even though Mujoco is installed.

I am using Windows 11, PyCharm Professional IDE and Python 3.7.
So my question is:
1, will the interactive viewer enable me to change some parameters in real-time while their effect is reflected in the viewer instantly. for example, I want to input the rotational velocity of a hinged link in real-time and see the model change its rotation speed in real-time. Is this possible? I am not using physics systems such as gravity, dynamics. But only kinematics to animate my robot.
Thank you.

[kevinzakka]

Did you upgrade to the latest version, i.e. v2.3.1.post1?

[tsgtdss583]

Did you upgrade to the latest version, i.e. v2.3.1.post1?

Yes I did. And the import error is gone.

[aftersomemath]

Regarding the changing of parameters, yes you should be able to do that. In this example the mjcb_control callback is used to change parameters in realtime. There are other ways to do it though including replacing the physics loop in viewer.py with a user defined implementation.

[umeiko]

I made up my own viewer via pygame lol. https://github.com/umeiko/Mujoco_Pygame_Interface

[saran-t]

I've just finished implementing a non-blocking passive viewer (similar to mujoco_py's viewer) in 230e278 which should sidestep any need for callbacks.

This should be shipped as part of the next release in the next few days. We'll post a discussion explaining how to beta test this thing.

[aftersomemath]

Looking forward to trying it!

[saran-t]

@aftersomemath IIRC you're not a macOS user, so you can try this out right now :)

On Windows and Linux, the new launch_passive call is rather trivial: see lines 304-310 here.

[aftersomemath]

Great!

Does the passive viewer handle locking the mjModel and mjData structures?

Or is locking disregarded since in theory the passive viewer doesn't modify the mjModel and mjData, so hopefully worse case scenario is some visual glitches in the viewer?

[saran-t]

It's technically racy, so you're right in that we are relying on the fact that the render thread only reads. In theory you may get some inconsistent rendering if the render is triggered in the middle of a step.

Locking isn't something that can be done transparently since only the user's script knows when it's modifying the physics. However, if races do become a problem it'll be relatively straightforward to expose an API for explicit locking.

In fact I plan to introduce that in the next-next release anyway when we get around to documenting the passive mode as an officially supported thing.

[yuvaltassa]

I can't imagine races being an issue. What, the renderer tries to draw contact frames in the same instant as the contacts change? The user changes the timestep from the GUI in the middle of an RK4 step?

Hard to imagine fatal stuff. I would wait until there is evidence of this really being a problem before giving people APIs that seem to imply that it is one 🙂

[saran-t]

I mean, if mjv_updateScene is called concurrently with mj_step (which is going to be quite common) then you'd literally be rendering a state that never exists.

I think what we have now will look plausibly correct but can never be trusted to show the exact correct state unless the user script is not updating the scene.

If you're trying to e.g. eyeball contacts in live, running simulation, then I'd say you're almost guaranteed to see the wrong thing.

[aftersomemath]

At least for me, launch_passive causes a segfault when interrupting the main "physics" thread with ctrl+c. My guess is that the viewer is trying to access data that doesn't exist after the physics thread exits.

An alternative lock-free approach is "double buffering". By reading from the visualized data mjData and putting the results in another mjData that is not being visualized. When mj_step is finished an atomic can be used to swap the mjData being visualized.

Many libraries and tools have "unsafe but fast" options that can be turned on, but usually they are not on by default (or at least can be turned off).

[saran-t]

launch_passive is now available as part of version 2.3.3. I'm going to close this Issue.

If anyone has comments or feedback relating to launch_passive, please reply to this discussion thread. If there is a bug in launch_passive or mjpython, please open a new Issue.