Yuchen Chen, Pengfei Ren*, Jingyu Wang, Haifeng Sun, Qi Qi, Jing Wang, Jianxin Liao

^aState Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

Abstract

3D hand reconstruction from RGB image has attracted a lot of attention due to its crucial role in human-computer interaction. Nevertheless, it is still challenging to perform 3D hand reconstruction under conditions of hand-object interaction due to severe mutual occlusion. Previous methods usually adopt fixed convolution kernel to extract features. We argue that simply sharing the static filter for all regions is impertinent, given that the occlusion degree varies across different regions, resulting in inconsistent visual representations. To address this issue, we proposed Region-aware Dynamic Filtering Network (RDFNet), which dynamically generates convolution kernels based on the features of different regions, thereby adaptively extracting region-related information. Furthermore, we introduce a dynamic receptive field selection mechanism to determine the most appropriate scale for the convolution kernel. For the severely occluded regions, larger receptive field is needed to capture semantic-related features, while the visible regions are mainly concerned with their own local pattern to accumulate spatial-related features and avoid the interference of irrelevant information. Our proposed RDFNet outperforms state-of-the-art methods by a large margin on several challenging hand-object interaction datasets.

Overview

RDFNet first extracts initial features from the backbone. Then, it take the initial features and perform region-aware dynamic feature refinement. Finally, decoder generates the MANO shape and pose parameters, which are propagated into the MANO layer and generate the final hand reconstruction result.

Bibtex

@inproceedings{chen2023region,
  title={Region-Aware Dynamic Filtering Network for 3D Hand Reconstruction.},
  author={Chen, Yuchen and Ren, Pengfei and Wang, Jingyu and Sun, Haifeng and Qi, Qi and Wang, Jing and Liao, Jianxin},
  booktitle={ECAI},
  pages={437--444},
  year={2023}
}