Problems when launching robot

[hao-l1]

🐛 Bug

When I try to launch the robot, an error occur that the subprocess died when command aborted.

HYDRA_FULL_ERROR=1 launch_robot.py robot_client=franka_hardware

/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/core/utils.py:146: UserWarning: register_resolver() is deprecated.
See omry/omegaconf#426 for migration instructions.

OmegaConf.register_resolver(name, f)
/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/plugins/config_source.py:190: UserWarning:
Missing @Package directive robot_client/franka_hardware.yaml in pkg://polymetis.conf.
See https://hydra.cc/docs/next/upgrades/0.11_to_1.0/adding_a_package_directive
warnings.warn(message=msg, category=UserWarning)
[2022-09-29 18:55:13,606][main][INFO] - Adding /home/haoli/anaconda3/envs/polymetis/lib/python3.8/build to $PATH
[2022-09-29 18:55:13,606][main][INFO] - Starting server
[2022-09-29 18:55:13,607][main][INFO] - Acquiring sudo...
"Acquired sudo."
[2022-09-29 18:55:13.843] [info] Using real time: true
[2022-09-29 18:55:13.843] [info] Using server address: 0.0.0.0:50051
Using 20MB as stack size
Using 80 as real-time thread priority
Using 0 as cpu_dma_latency
Disabled sbrk...
Disabled mmap...
Locked memory...
Initialized pthread...
Finished setting stacksize...
Finished setting scheduling policy & priority...
Creating thread...
Started realtime thread.
[2022-09-29 18:55:14.615] [info] Server listening on 0.0.0.0:50051
[2022-09-29 18:55:14,642][main][INFO] - Starting robot client...
/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/utils.py:32: UserWarning: OmegaConf.is_none() is deprecated, see omry/omegaconf#547
if OmegaConf.is_none(config):
[2022-09-29 18:55:14,710][polymetis.robot_client.executable_robot_client][INFO] - === Config: ===
robot_ip: 172.16.0.2
control_ip: 0.0.0.0
control_port: 50051
readonly: false
mock: false
use_real_time: true
hz: 1000
num_dofs: 7
exec: franka_panda_client
robot_client_metadata_path: /tmp/tmp6tc7yc2q
limit_rate: true
lpf_cutoff_frequency: 100
limits:
cartesian_pos_upper:

• 1.0
• 0.4
• 1.0
  cartesian_pos_lower:
• 0.1
• -0.4
• -0.05
  joint_pos_upper:
• 2.8
• 1.66
• 2.8
• -0.17
• 2.8
• 3.65
• 2.8
  joint_pos_lower:
• -2.8
• -1.66
• -2.8
• -2.97
• -2.8
• 0.08
• -2.8
  joint_vel:
• 2.075
• 2.075
• 2.075
• 2.075
• 2.51
• 2.51
• 2.51
  elbow_vel: 2.075
  joint_torques:
• 86.0
• 86.0
• 86.0
• 86.0
• 11.5
• 11.5
• 11.5
  collision_behavior:
  lower_torque:
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
  upper_torque:
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
  lower_force:
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
  upper_force:
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
• 40.0
  safety_controller:
  is_active: true
  margins:
  cartesian_pos: 0.05
  joint_pos: 0.2
  joint_vel: 0.5
  stiffness:
  cartesian_pos: 200.0
  joint_pos: 50.0
  joint_vel: 20.0

[2022-09-29 18:55:14,711][polymetis.robot_client.executable_robot_client][INFO] - === Executing client at /home/haoli/anaconda3/envs/polymetis/bin/franka_panda_client ===
Using 20MB as stack size
Using 80 as real-time thread priority
Using 0 as cpu_dma_latency
Disabled sbrk...
Disabled mmap...
Locked memory...
Initialized pthread...
Finished setting stacksize...
Finished setting scheduling policy & priority...
Creating thread...
Started realtime thread.
[2022-09-29 18:55:14.770] [info] ==== Initializing new RobotClient... ====
[2022-09-29 18:55:14.903] [info] Success.
[2022-09-29 18:55:14.904] [info] Connecting to Franka Emika...
Traceback (most recent call last):
File "/home/haoli/anaconda3/envs/polymetis/bin/launch_robot.py", line 94, in
main()
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/_internal/utils.py", line 201, in run_and_report
raise ex
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/hydra/core/utils.py", line 127, in run_job
ret.return_value = task_function(task_cfg)
File "/home/haoli/anaconda3/envs/polymetis/bin/launch_robot.py", line 90, in main
client.run()
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/site-packages/polymetis/robot_client/executable_robot_client.py", line 78, in run
subprocess.run(
File "/home/haoli/anaconda3/envs/polymetis/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sudo', 'env', '"PATH=$PATH"', '/home/haoli/anaconda3/envs/polymetis/bin/franka_panda_client', '/tmp/tmpsqsvo0ia']' died with <Signals.SIGABRT: 6>.
[2022-09-29 18:55:15,073][main][INFO] - Using sudo to kill subprocess with pid 3458, pgid 3458...

To Reproduce

Steps to reproduce the behavior:

1. use real time kernel 5.11.4
2. create a conda env for polymetis
3. activate env and run launch_robot.py robot_client=franka_hardware

Expected behavior

Is it because I use the wrong real time kernel? I met some problem when building the kernel 5.11.0, so I just download the kernel that others' already built.

Environment

• Version (Git commit): latest
• Environment (which devfair/robodev/laptop) ubuntu 20.04

Additional context

[1heart]

Can you verify that the communication test works before running Polymetis? This will determine whether it's a Polymetis-specific issue or (as you suspect) a real-time kernel issue.

The libfranka communication test is bundled in the Polymetis conda package.

Also, our new repo is at fairo. I would suggest adding future Issues there to centralize our discussion.