Conference proceeding
Distilling CLIP with Dual Guidance for Learning Discriminative Human Body Shape Representation
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 256-266
16 Jun 2024
Abstract
Person Re-ldentification (ReID) holds critical importance in computer vision with pivotal applications in public safety and crime prevention. Traditional ReID methods, reliant on appearance attributes such as clothing and color, en-counter limitations in long-term scenarios and dynamic environments. To address these challenges, we propose CLIP3DReID, an innovative approach that enhances person ReID by integrating linguistic descriptions with visual per-ception, leveraging pretrained CLIP model for knowledge distillation. Our method first employs CLIP to automatically label body shapes with linguistic descriptors. We then ap-ply optimal transport theory to align the student model's local visual features with shape-aware tokens derived from CLIP's linguistic output. Additionally, we align the student model's global visual features with those from the CLIP image encoder and the 3D SMPL identity space, fostering enhanced domain robustness. CLIP3DReID notably excels in discerning discriminative body shape features, achieving state-of-the-art results in person ReID. Our approach rep-resents a significant advancement in ReID, offering robust solutions to existing challenges and setting new directions for future research. Code is available.
Metrics
5 Record Views
6 citations in Scopus
Details
- Title
- Distilling CLIP with Dual Guidance for Learning Discriminative Human Body Shape Representation
- Creators
- Feng Liu - Drexel University, Computer Science (Computing)Minchul Kim - Michigan State UniversityZhiyuan Ren - Michigan State UniversityXiaoming Liu - Michigan State University
- Publication Details
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 256-266
- Publisher
- IEEE
- Grant note
- Office of the Director of National Intelligence (ODNI) (10.13039/100011038) 2022-21102100004 / Intelligence Advanced Research Projects Activity (IARPA) (10.13039/100011039)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Scopus ID
- 2-s2.0-85216076253
- Other Identifier
- 991021906502204721