Parameter reconstruction of 3D textured human mesh based on single image

Journal Information

J. Hefei Univ. Tech. (Nat. Sci.), Mar.2024, Vol.47 No.3: 347-353

DOI: 10.3969/j.issn.1003-5060.2024.03.010

Authors

XING Yan $ ^{1} $, XU Dong $ ^{1} $, HONG Peilin $ ^{2} $, TAN Jieqing $ ^{1} $

(1. School of Mathematics, Hefei University of Technology, Hefei 230601, China; 2. School of Medical Information Engineering, Anhui University of Chinese Medicine, Hefei 230012, China)

Abstract and Keywords

Abstract: Aiming at the problem of 3D human reconstruction in computer vision, an end-to-end network framework is proposed to reconstruct accurate 3D human mesh with texture from a single color image under the hybrid supervision of 3D and 2D. In this paper, four encoders are used to extract shape and pose features, texture features, illumination parameters and camera parameters respectively. And the features obtained are sent to the 3D regression module to iteratively infer the parameters of the 3D human model. Texture parameters are fed into the texture decoder network to obtain texture maps. The learned human model parameters can be transformed into the 3D human mesh. For the setting of loss function, the difference between the predicted human mesh vertex and the ground truth is used for 3D supervision. The 2D rendering loss is calculated by predicted camera parameters, illumination parameters and mapped texture. The 2D joint reprojection loss is calculated by projecting the 3D joints to the 2D joints and then comparing with the ground truth. The discriminator of generative adversarial network (GAN) is used to make the rendered images more realistic. The qualitative and quantitative experimental results show that the proposed method achieves comparable performance with some state-of-the-art 3D human reconstruction methods. Moreover, the reconstructed 3D human mesh possesses the corresponding texture map according to the input human image.

Keywords: 3D human reconstruction; deep learning; skinned multi-person linear (SMPL) model;

作者信息

邢燕 $ ^{1} $，徐冬 $ ^{1} $，洪沛霖 $ ^{2} $，檀结庆 $ ^{1} $

(1. 合肥工业大学数学学院, 安徽合肥 230601; 2. 安徽中医药大学医药信息工程学院, 安徽合肥 230012)

摘要和关键词

摘要: 针对计算机视觉中的三维人体重建问题,文章提出一种端到端的的网络框架,在三维和二维混合监督下,从单幅彩色图像重建带纹理信息的精准三维人体网格。使用4个编码器分别提取形状姿态特征、纹理特征、光照参数和像机参数,得到的图像特征被送入三维回归模块,迭代推断出三维人体参数;纹理参数送入纹理解码器网络得到纹理图;学习到的人体参数可转化为三维人体网格;对于损失函数的设置,预测的人体网格顶点与真实顶点的差值用来进行三维监督;通过预测的像机参数、光照参数和纹理计算二维渲染损失;通过三维关节投射得到的二维关节与图像上的二维关节真值计算二维关节重投影损失;生成对抗网络的鉴别器使得渲染图像更加真实。该文方法与现有的三维人体重建方法相比具有竞争力,而且重建的三维人体网格带有纹理信息。

关键词: 三维人体重建；深度学习；蒙皮多人线性（SMPL）模型；形状姿态；纹理

Funding

国家自然科学基金资助项目（62172135）；合肥工业大学校级教研资助项目（KCSZ2022034）；安徽中医药大学教研重点资助项目（2020xjjy_zd005）

合肥工业大学学报(自然科学版)

Navigation