Domain Randomization

[PURANJAY14]

Hi,
I have 5 different humanoid body shapes stored in 5 different .xml files. I want to apply domain randomization where each episode can start with either of the 5 humanoids. How can I implement this using skrl ?

[Toni-SM]

HI @PURANJAY14

Could you please provide more details, such as the environment interface (e.g. gymnasium, Omniverse Isaac Gym, etc.), the number of parallel environments (if any), or other relevant information?

What about applying any of the 5 humanoids when the episode is finished and the environment is reset in the task/environment file itself?

[PURANJAY14]

Isaac Gym interface with 4096 parallel environments. The pipeline is same as that of deafult humanoid-ppo. How do I modify this default humanoid-ppo to incorporate domain randomization prescribed above(where we start a new episode with a randomly assigned humanoid).

[Toni-SM]

Hi @PURANJAY14

You can modify the following lines to set a different humanoid, previously loaded in the __init__ method (for faster execution), for each reset environment.

https://github.com/NVIDIA-Omniverse/IsaacGymEnvs/blob/665c32170d84b4be66722eea405a1e08b6e7f761/isaacgymenvs/tasks/humanoid.py#L258-L259

[PURANJAY14]

humanoid have different shapes but same state and action space but if I create a new actor using the following lines of code

asset_root = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../assets')
asset_files = ["mjcf/nv_humanoid.xml", "mjcf/nv_humanoid_hands.xml"]
# Randomly choose an asset file from the list
asset_file = random.choice(asset_files)
asset_path = os.path.join(asset_root, asset_file)
asset_root = os.path.dirname(asset_path)
asset_file = os.path.basename(asset_path)
asset_options = gymapi.AssetOptions()
asset_options.angular_damping = 0.01
asset_options.max_angular_velocity = 100.0
# Note - DOF mode is set in the MJCF file and loaded by Isaac Gym
asset_options.default_dof_drive_mode = gymapi.DOF_MODE_NONE
humanoid_asset = self.gym.load_asset(self.sim, asset_root, asset_file, asset_options)
start_pose = gymapi.Transform()
start_pose.p = gymapi.Vec3(*get_axis_params(1.34, self.up_axis_idx))
start_pose.r = gymapi.Quat(0.0, 0.0, 0.0, 1.0)

for env_id in env_ids:
            handle = self.gym.create_actor(self.envs[env_id], humanoid_asset, start_pose, "humanoid", env_id, 0, 0)
            for j in range(self.num_bodies):
                self.humanoid_handles[env_id]=handle

(added just before the positions = torch_rand_float(-0.2, 0.2, (len(env_ids), self.num_dof), device=self.device)) gives the following error

[Warning] [carb.gym.plugin] Incorrect actor creation order.
All actors must be added to an env before creating the next env.
Adding actors to envs out of order can result in errors.

To suppress this warning, set enable_actor_creation_warning to False in SimParams.

[Error] [carb.gym.plugin] Function GymGetActorDofStates cannot be used with the GPU pipeline after simulation starts. Please use the tensor API if possible. See docs/programming/tensors.html for more info.
Segmentation fault

[Toni-SM]

Hi @PURANJAY14

Now that I have more information, I think that in your case you can instantiate all the humanoids (create the actors) as follows (in the _create_envs method)

custom_humanoid_assets = ... # list of 5 assets

# ...

for i in range(self.num_envs):
    # create env instance
    env_ptr = self.gym.create_env(
        self.sim, lower, upper, num_per_row
    )
    # custom humanoids
    handle = self.gym.create_actor(env_ptr, custom_humanoid_assets[i % 5], start_pose, "humanoid", i, 0, 0)