Image-based 3D reconstruction and articulation of the human body shape and its use in the creation of virtual fitting rooms

Chenxi Li

doi:10.17918/00000196

Image-based 3D shape reconstruction is essential for 3D modeling in recognition, virtual reality, generation of video games, 3D animation, and in the creation of 3D virtual fitting rooms - the sought-after application in this dissertation. In this work, we propose a novel reconstruction method that uses a generic 3D model that roughly controls the overall reconstruction shape which in turn, reduces the number of 2D images needed to just two (frontal and profile) for obtaining a 3D point model. The generic model consists of a massive set of 3D points that lack saliency, topological meaning, and relational interconnections. This is also true about points in the two canonical images. In this work, we propose finding a small set of interconnected ordered intrinsic salient points (control points) residing on the silhouette of the projections of the generic model onto the two image spaces, as well as finding those for the canonical images. Points on the generic model and the images are automatically given saliency with order and interconnections through loop-subdivision operating on those control points and their equivalent points on the canonical images. The transformation of the generic model into the specific object is done through morphing the points on the generic model to follow and be consistent with their equivalent points on the canonical images. The control points on the canonical images are automatically obtained using an active shape model (ASM) that modifies an average trained control point set attained using an offline training, to account for class shape variability and boundary regularization guaranteeing the control points to reside on the silhouette (boundary) of the individual. The reconstructed 3D model is furtherly refined through smoothing the reconstructed surface, while fixing local holes using a principle component analysis (PCA) decomposition. This reconstruction method is convenient, simple, efficient, and results in an average geometric error of less than 0.5% over 700 diverse body models from the CAESAR dataset. We then implement the contour-based articulated model to the personalized 3D model for pose recovery. The evolution of the model through the articulation process is captured in a video clip. 3D transformations are found and applied to the 3D model on its parts so that the projections of the 3D model 'match' those observed in the video sequence at corresponding frames. This is done by minimizing the error between the frontal projection body region points and the target points from the image for each independent moving part, with sub-resolution recovery errors. Once the personalized 3D model is obtained and the articulated model is implemented, the algorithm simulates how garments virtually appear on the reconstructed person under different poses. Different types of clothes are classified as tight or loose, where for the first type of garments, the garment is mapped onto the person's body, whereas for the second type, the mapping adjusts to accommodate the loose nature of the garment, and its fitting on the body of the reconstructed person model. This creates a virtual in-home fitting room for garment fitting, which is timely and is a needed application that would go hand in hand with online clothes shopping, allowing the trying of garments at home with changing poses before ordering them, and can be updated as often as necessary to accommodate changes in physique, weight, aging, etc. This is also done with minimum requirements from the user aside from taking two images and a video of pose articulations using their cam or smart phone, and stands as a useful, needed, and attractive App.

Image-based 3D reconstruction and articulation of the human body shape and its use in the creation of virtual fitting rooms

Files and links (1)

Abstract

Metrics

Details

Image-based 3D reconstruction and articulation of the human body shape and its use in the creation of virtual fitting rooms

Files and links (1)

Abstract

Metrics

Details

Drexel University Social media