Skip to content

Latest commit

 

History

History
72 lines (59 loc) · 2.32 KB

README.md

File metadata and controls

72 lines (59 loc) · 2.32 KB
Raw

IPNet

This repository is an official implementation of the AAAI 2023 paper "Two Heads are Better than One: Image-Point Cloud Network for Depth-Based 3D Hand Pose Estimation".

Two Heads are Better than One: Image-Point Cloud Network for Depth-Based 3D Hand Pose Estimation

Pengfei Ren, Chenyu Chen, Jiachang Hao, Haifeng Sun, Qi Qi, Jingyu Wang, Jianxin Liao

Installation

Prerequisites

  • Python >= 3.8
  • PyTorch >= 1.10
  • CUDA (tested with cuda11.3)
  • Other dependencies described in requirements.txt
  • Install point operation 
    pip install pointnet2_ops_lib/.
  • Install Manopth

Install MANO

  • Go to MANO website
  • Download Models and Code (the downloaded file should have the format mano_v*_*.zip).
  • unzip and copy the models/MANO_RIGHT.pkl into the MANO folder
  • Your folder structure should look like this:
code/
  MANO/
    MANO_RIGHT.pkl

Prepare Dataset

DexYCB

  • Download and decompress DexYCB
  • Modify the root_dir in config.py according to your setting.
  • Generate json file for data loading (dataloader/DEXYCB2COCO.py)
  • In order to speed up the training, you need to generate the hand mesh corresponding to each image according to the MANO annotation.
  • Your folder structure should look like this:
DexYCB/
  mesh/
    20200709-subject-01/
        20200709_153548/
            932122062010/
                mesh_000000.txt
                ...
    ...
  20200709-subject-01/
  20200813-subject-02/
  ...

NYU

  • Download and decompress NYU
  • Modify the root_dir in config.py according to your setting.

Train

DexYCB

python train_ho.py

NYU

python train.py

Remember to change the dataset name in config.py accordingly.