Preprint
From 3D Pose to Prose: Biomechanics-Grounded Vision–Language Coaching
ArXiv.org
27 Mar 2026
Abstract
We present BioCoach, a biomechanics-grounded vision–language framework for fitness coaching from streaming video. BioCoach fuses visual appearance and 3D skeletal kinematics, through a novel three-stage pipeline: an exercise-specific degree-of-freedom selector that focuses analysis on salient joints; a structured biomechanical context that pairs individualized morphometrics with cycle and constraint analysis; and a vision–biomechanics conditioned feedback module that applies cross-attention to generate precise, actionable text. Using parameter-efficient training that freezes the vision and language backbones, BioCoach yields transparent, personalized reasoning rather than pattern matching. To enable learning and fair evaluation, we augment QEVD-fit-coach with biomechanics-oriented feedback to create QEVD-bio-fit-coach, and we introduce a biomechanics-aware LLM judge metric. BioCoach delivers clear gains on QEVD-bio-fit-coach across lexical and judgment metrics while maintaining temporal triggering; on the original QEVD-fit-coach, it improves text quality and correctness with near-parity timing, demonstrating that explicit kinematics and constraints are key to accurate, phase-aware coaching.
Metrics
1 Record Views
Details
- Title
- From 3D Pose to Prose: Biomechanics-Grounded Vision–Language Coaching
- Creators
- Yuyang Ji - Drexel UniversityYixuan Shen - Drexel UniversityShengjie Zhu - Michigan State UniversityYu Kong - Michigan State UniversityFeng Liu - Drexel University
- Publication Details
- ArXiv.org
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science
- Other Identifier
- 991022172873204721