Files and links (1)
SubmittedarXiv.org - Non-exclusive license to distribute, Open
Conference proceeding
3D Human Motion Generation from the Text Via Gesture Action Classification and the Autoregressive Model
2022 IEEE International Conference on Image Processing (ICIP), pp 1036-1040
16 Oct 2022
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
In this paper, a deep learning-based model for 3D human motion generation from the text is proposed via gesture action classification and an autoregressive model. The model focuses on generating special gestures that express human thinking, such as waving and nodding. To achieve the goal, the proposed method predicts expression from the sentences using a text classification model based on a pretrained language model and generates gestures using the gate recurrent unit-based autoregressive model. Especially, we proposed the loss for the embedding space for restoring raw motions and generating intermediate motions well. Moreover, the novel data augmentation method and stop token are proposed to generate variable length motions. To evaluate the text classification model and 3D human motion generation model, a gesture action classification dataset and action-based gesture dataset are collected. With several experiments, the proposed method successfully generates perceptually natural and realistic 3D human motion from the text. Moreover, we verified the effectiveness of the proposed method using a public-available action recognition dataset to evaluate cross-dataset generalization performance.
Metrics
Details
- Title
- 3D Human Motion Generation from the Text Via Gesture Action Classification and the Autoregressive Model
- Creators
- Gwantae Kim - Korea UniversityYoungsuk Ryu - Korea UniversityJunyeop Lee - Korea UniversityDavid K. Han - Drexel UniversityJeongmin Bae - DMLab,Seoul,South KoreaHanseok Ko - Korea University
- Publication Details
- 2022 IEEE International Conference on Image Processing (ICIP), pp 1036-1040
- Publisher
- IEEE
- Number of pages
- 5
- Grant note
- Korea University (10.13039/501100002642)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Web of Science ID
- WOS:001058109501028
- Scopus ID
- 2-s2.0-85146654416
- Other Identifier
- 991021930832204721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Engineering, Electrical & Electronic