Isaac Sim 4.0 support, Kinematics API doc, Windows support

CHANGELOG.md

@@ -10,16 +10,58 @@ its affiliates is strictly prohibited.
 -->
 # Changelog
 
-## Latest Commit
+## Version 0.7.4
+
+### Changes in Default Behavior
+
+- Cuda graph capture of optimization iterations resets solver before recording.
+- ``join_path(a, b)`` now requires ``a`` to not have a trailing slash to make the file compatible with Windows.
+- Drop examples support for Isaac Sim < 4.0.0.
+- asset_root_path can be either empty string or None.
+- Order of variables in ``SelfCollisionKinematicsConfig`` has changed. Unused variables
+moved to bottom.
+- Remove requirement of warmup for using ``offset_waypoint`` in ``PoseCost``.
+
+### New Features
+
+- Interpolated metrics calculation now recreates cuda graph if interpolation steps exceed existing buffer size.
+- Add experimental ``CUDAGraph.reset`` usage as ``cuda>=12.0`` is not crashing when an existing captured CUDAGraph is freed and recaptured with new memory pointers. Try this experimental feature by
+setting an environment variable ``export CUROBO_TORCH_CUDA_GRAPH_RESET=1``. This feature will allow for changing the problem type in ``motion_gen`` and ``ik_solver`` without requiring recreation of the class.
+- Add partial support for Windows.
+- Add Isaac Sim 4.0.0 docker support.
+- Examples now work with Isaac Sim 4.0.0.
+- Add XRDF support.
+- Add curobo.types.file_path.ContentPath to store paths for files representing robot and world. This
+improves development on top of cuRobo with custom robots living external of cuRobo library.
+- Add attach external objects to robot link API to CudaRobotModel.
+- Add MotionGenStatus.DT_EXCEPTION to report failures due to trajectory exceeding user specified
+maximum trajectory dt.
+- Add reading of end-effector mesh if available when rendering trajectory with ``UsdHelper``, also
+supports goalset rendering.
+- Kinematics module (`curobo.cuda_robot_model`) has complete API documentation.
 
 ### BugFixes & Misc.
-- Add support for older warp versions (<1.0.0) as it's not possible to run older isaac sim with
-newer warp versions.
+
+- Minor documentation fixes to install instructions.
+- Add support for older warp versions (<1.0.0) as it's not possible to run older isaac sim with newer warp versions.
 - Add override option to mpc dataclass.
 - Fix bug in ``PoseCost.forward_pose()`` which caused ``torch_layers_example.py`` to fail.
 - Add warp constants to make module hash depend on robot dof, for modules that generate runtime
 warp kernels. This fixes issues using cuRobo in isaac sim.
 - Add ``plan_config.timeout`` check to ``plan_single_js()``.
+- Recreation of interpolation buffer now copies the joint names from raw trajectory.
+- Fix bug in running captured cuda graph on deleted memory pointers
+when getting metrics on interpolated trajectory
+- Change order of operations in cuda graph capture of particle opt to get correct results
+during graph capture phase.
+- Franka Panda now works in Isaac Sim 4.0.0. The fix was to add inertial parameters to all links in
+the urdf.
+- Create new instances of rollouts in wrap classes to ensure cuda graph rollouts are not
+accidentally used in other pipelines.
+- Add cuda graph check for ``get_metrics``.
+- Remove aligned address assumption for float arrays inside kernel (local memory).
+- Add check for existing warp kernel in a module before creating a new one to avoid corruption of
+existing cuda graphs.
 
 ## Version 0.7.3
 

benchmark/ik_benchmark.py

@@ -109,7 +109,7 @@ def run_full_config_collision_free_ik(
     parser.add_argument(
         "--save_path",
         type=str,
-        default=".",
+        default=None,
         help="path to save file",
     )
     parser.add_argument(
@@ -132,9 +132,9 @@ def run_full_config_collision_free_ik(
     )
     args = parser.parse_args()
 
-    b_list = [1, 10, 100, 500, 2000][:]
+    b_list = [1, 10, 100, 2000][-1:]
 
-    robot_list = get_motion_gen_robot_list() + get_multi_arm_robot_list()[:2]
+    robot_list = get_motion_gen_robot_list()
     world_file = "collision_test.yml"
 
     print("running...")
@@ -187,15 +187,21 @@ def run_full_config_collision_free_ik(
             data["Orientation-Error-Collision-Free-IK"].append(q_err_c)
 
             data["Collision-Free-IK-time(ms)"].append(dt_cu_ik_cfree * 1000.0)
-    write_yaml(data, join_path(args.save_path, args.file_name + ".yml"))
+
+    if args.save_path is not None:
+        file_path = join_path(args.save_path, args.file_name)
+    else:
+        file_path = args.file_name
+
+    write_yaml(data, file_path + ".yml")
 
     try:
         # Third Party
         import pandas as pd
 
         df = pd.DataFrame(data)
         print("Reported errors are 98th percentile")
-        df.to_csv(join_path(args.save_path, args.file_name + ".csv"))
+        df.to_csv(file_path + ".csv")
         try:
             # Third Party
             from tabulate import tabulate

docker/aarch64.dockerfile

@@ -124,15 +124,16 @@ RUN pip3 install trimesh \
   numpy-quaternion \
   networkx \
   pyyaml \
-  rospkg \ 
+  rospkg \
   rosdep \
   empy
 
 # Add cache date to avoid using cached layers older than this
-ARG CACHE_DATE=2024-04-11
+ARG CACHE_DATE=2024-07-19
+
 
 # install warp:
-# 
+#
 RUN pip3 install warp-lang
 
 # install curobo:
@@ -165,7 +166,7 @@ RUN cd /pkgs && git clone https://github.com/nvlabs/nvblox_torch.git && \
     sh install.sh $(python3 -c 'import torch.utils; print(torch.utils.cmake_prefix_path)') && \
     python3 -m pip install -e .
 
-  
+
 RUN python3 -m pip install "robometrics[evaluator] @ git+https://github.com/fishbotics/robometrics.git"
 
 # upgrade typing extensions:

docker/build_dev_docker.sh

@@ -35,4 +35,4 @@ fi
 echo $input_arg
 echo $USER_ID
 
-docker build --build-arg USERNAME=$USER --build-arg USER_ID=${USER_ID} --build-arg IMAGE_TAG=$input_arg -f $user_dockerfile --tag curobo_docker:user_$input_arg . 
+docker build --build-arg USERNAME=$USER --build-arg USER_ID=${USER_ID} --build-arg IMAGE_TAG=$input_arg -f $user_dockerfile --tag curobo_docker:user_$input_arg .

docker/build_docker.sh

@@ -32,16 +32,11 @@ if [ -z "$input_arg" ]; then
     fi
 fi
 
-if [ "$input_arg" == "isaac_sim_2022.2.1" ]; then
-    echo "Building Isaac Sim docker"
-    dockerfile="isaac_sim.dockerfile"
-    image_tag="isaac_sim_2022.2.1"
-    isaac_sim_version="2022.2.1"
-elif [ "$input_arg" == "isaac_sim_2023.1.0" ]; then
+if [ "$input_arg" == "isaac_sim_4.0.0" ]; then
     echo "Building Isaac Sim headless docker"
     dockerfile="isaac_sim.dockerfile"
-    image_tag="isaac_sim_2023.1.0"
-    isaac_sim_version="2023.1.0"
+    image_tag="isaac_sim_4.0.0"
+    isaac_sim_version="4.0.0"
 elif [ "$input_arg" == "x86" ]; then
     echo "Building for X86 Architecture"
     dockerfile="x86.dockerfile"
@@ -63,11 +58,11 @@ fi
 #        "nvidia": {
 #            "path": "/usr/bin/nvidia-container-runtime",
 #            "runtimeArgs": []
-#         } 
+#         }
 #    },
 #    "default-runtime": "nvidia" # ADD this line (the above lines will already exist in your json file)
 # }
-# 
+#
 echo "${dockerfile}"
 
-docker build --build-arg ISAAC_SIM_VERSION=${isaac_sim_version} -t curobo_docker:${image_tag} -f ${dockerfile} . 
+docker build --build-arg ISAAC_SIM_VERSION=${isaac_sim_version} -t curobo_docker:${image_tag} -f ${dockerfile} .

docker/isaac_sim.dockerfile

@@ -11,7 +11,7 @@
 ARG DEBIAN_FRONTEND=noninteractive
 ARG BASE_DIST=ubuntu20.04
 ARG CUDA_VERSION=11.4.2
-ARG ISAAC_SIM_VERSION=2022.2.1
+ARG ISAAC_SIM_VERSION=4.0.0
 
 
 FROM nvcr.io/nvidia/isaac-sim:${ISAAC_SIM_VERSION} AS isaac-sim
@@ -20,7 +20,7 @@ FROM nvcr.io/nvidia/cudagl:${CUDA_VERSION}-devel-${BASE_DIST}
 
 
 # this does not work for 2022.2.1
-#$FROM nvcr.io/nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-${BASE_DIST} 
+#$FROM nvcr.io/nvidia/cuda:${CUDA_VERSION}-cudnn8-devel-${BASE_DIST}
 
 LABEL maintainer "User Name"
 
@@ -102,7 +102,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 RUN wget -q --show-progress \
     --progress=bar:force:noscroll \
     https://sdk.lunarg.com/sdk/download/${VULKAN_SDK_VERSION}/linux/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.gz \
-    -O /tmp/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.gz \ 
+    -O /tmp/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.gz \
     && echo "Installing Vulkan SDK ${VULKAN_SDK_VERSION}" \
     && mkdir -p /opt/vulkan \
     && tar -xf /tmp/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.gz -C /opt/vulkan \
@@ -118,7 +118,7 @@ RUN wget -q --show-progress \
     && rm /tmp/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.gz && rm -rf /opt/vulkan
 
 
-# Setup the required capabilities for the container runtime    
+# Setup the required capabilities for the container runtime
 ENV NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=all
 
 # Open ports for live streaming
@@ -157,16 +157,15 @@ ENV TORCH_CUDA_ARCH_LIST="7.0+PTX"
 # create an alias for omniverse python
 ENV omni_python='/isaac-sim/python.sh'
 
-RUN echo "alias omni_python='/isaac-sim/python.sh'" >> /.bashrc
+RUN echo "alias omni_python='/isaac-sim/python.sh'" >> ~/.bashrc
 
 
 # Add cache date to avoid using cached layers older than this
 ARG CACHE_DATE=2024-04-11
 
-RUN $omni_python -m pip install "robometrics[evaluator] @ git+https://github.com/fishbotics/robometrics.git"
 
 # if you want to use a different version of curobo, create folder as docker/pkgs and put your
-# version of curobo there. Then uncomment below line and comment the next line that clones from 
+# version of curobo there. Then uncomment below line and comment the next line that clones from
 # github
 
 # COPY pkgs /pkgs

docker/start_docker.sh

@@ -56,6 +56,7 @@ elif [ "$input_arg" == "aarch64" ]; then
 
 elif [[ "$input_arg" == *isaac_sim* ]] ; then
 
+
     docker run --name container_$input_arg --entrypoint bash -it --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
         --privileged \
         -e "PRIVACY_CONSENT=Y" \

docker/user.dockerfile

@@ -15,7 +15,7 @@ FROM curobo_docker:${IMAGE_TAG}
 # Set variables
 ARG USERNAME
 ARG USER_ID
-ARG CACHE_DATE=2024-04-25
+ARG CACHE_DATE=2024-07-19
 
 # Set environment variables
 

docker/user_isaac_sim.dockerfile

@@ -10,11 +10,13 @@
 ##
 
 # Check architecture and load:
-ARG IMAGE_TAG 
+ARG IMAGE_TAG
 FROM curobo_docker:${IMAGE_TAG}
 # Set variables
 ARG USERNAME
 ARG USER_ID
+ARG CACHE_DATE=2024-07-19
+
 
 # Set environment variables
 
@@ -44,7 +46,7 @@ RUN chown -R $USERNAME:users /isaac-sim/exts/omni.isaac.synthetic_recorder/
 RUN chown -R $USERNAME:users /isaac-sim/kit/exts/omni.gpu_foundation
 RUN mkdir -p /home/$USERNAME/.cache && cp -r /root/.cache/* /home/$USERNAME/.cache && chown -R $USERNAME:users /home/$USERNAME/.cache
 RUN mkdir -p /isaac-sim/kit/data/documents/Kit && mkdir -p /isaac-sim/kit/data/documents/Kit/apps/Isaac-Sim/scripts/ &&chown -R $USERNAME:users /isaac-sim/kit/data/documents/Kit /isaac-sim/kit/data/documents/Kit/apps/Isaac-Sim/scripts/
-RUN mkdir -p /home/$USERNAME/.local 
+RUN mkdir -p /home/$USERNAME/.local
 
 
 RUN echo "alias omni_python='/isaac-sim/python.sh'" >> /home/$USERNAME/.bashrc

docker/x86.dockerfile

@@ -78,7 +78,7 @@ ENV TORCH_CUDA_ARCH_LIST "7.0+PTX"
 ENV LD_LIBRARY_PATH="/usr/local/lib:${LD_LIBRARY_PATH}"
 
 # Add cache date to avoid using cached layers older than this
-ARG CACHE_DATE=2024-04-25
+ARG CACHE_DATE=2024-07-19
 
 
 RUN pip install "robometrics[evaluator] @ git+https://github.com/fishbotics/robometrics.git"

examples/isaac_sim/batch_collision_checker.py

@@ -9,6 +9,12 @@
 # its affiliates is strictly prohibited.
 #
 
+try:
+    # Third Party
+    import isaacsim
+except ImportError:
+    pass
+
 
 # Third Party
 import torch

examples/isaac_sim/batch_motion_gen_reacher.py

@@ -10,6 +10,12 @@
 #
 
 
+try:
+    # Third Party
+    import isaacsim
+except ImportError:
+    pass
+
 # Third Party
 import torch
 

examples/isaac_sim/collision_checker.py

@@ -9,6 +9,12 @@
 # its affiliates is strictly prohibited.
 #
 
+try:
+    # Third Party
+    import isaacsim
+except ImportError:
+    pass
+
 
 # Third Party
 import torch

examples/isaac_sim/constrained_reacher.py

@@ -9,6 +9,12 @@
 # its affiliates is strictly prohibited.
 #
 
+try:
+    # Third Party
+    import isaacsim
+except ImportError:
+    pass
+
 # Third Party
 import torch
 
@@ -156,7 +162,6 @@
         tensor_args,
         collision_checker_type=CollisionCheckerType.MESH,
         collision_cache={"obb": n_obstacle_cuboids, "mesh": n_obstacle_mesh},
-        velocity_scale=0.75,
         interpolation_dt=0.02,
         ee_link_name="right_gripper",
     )
@@ -180,7 +185,11 @@
     cmd_plan = None
     articulation_controller = robot.get_articulation_controller()
     plan_config = MotionGenPlanConfig(
-        enable_graph=False, enable_graph_attempt=4, max_attempts=2, enable_finetune_trajopt=True
+        enable_graph=False,
+        enable_graph_attempt=4,
+        max_attempts=2,
+        enable_finetune_trajopt=True,
+        time_dilation_factor=0.5,
     )
 
     plan_idx = 0