Imagine a world where your personal trainer is not just a human expert but an AI-powered coach, providing real-time feedback on your exercise form to prevent injuries and enhance performance. This futuristic scenario is not as far-fetched as it may seem, thanks to the innovative work of researchers from Drexel University and Michigan State University. Their creation, BioCoach, is a prototype program that combines artificial intelligence, computer vision, and biomechanical modeling to offer personalized exercise guidance, a significant step forward from the generic feedback often provided by fitness apps.
The need for such a system became evident during the COVID-19 pandemic, when many people turned to at-home workouts. The U.S. Consumer Product Safety Commission reported a sharp rise in exercise-related injuries during this period, highlighting the importance of proper form and guidance, especially for those without regular access to coaches or trainers.
The BioCoach Advantage
BioCoach aims to bridge this gap by providing timely, specific cues grounded in body motion. It achieves this by analyzing video footage of exercises and offering detailed feedback, a feature that has proven challenging for most fitness coaching apps. The program's unique integration of computer vision and biomechanical modeling allows it to identify relevant joints and provide feedback based on concrete movement issues, explaining why they matter.
Preparing BioCoach for Action
The team began by enhancing the publicly available Qualcomm Exercise Video Dataset (QEVD), which includes hundreds of hours of exercise footage with time-stamped coaching feedback. They re-annotated the dataset with more detailed biomechanical targets and added rationale for the guidance, creating a more comprehensive and informative dataset. This dataset, with over 2,400 additional notes, was used to train and test BioCoach, ensuring it could provide accurate and timely feedback.
How BioCoach Works
BioCoach employs two complementary streams of information to analyze each video. One stream uses a 3D convolutional neural network to capture visual appearance and motion patterns, while the other estimates 3D skeletal movements and body shape, providing information on joint angles, ranges of motion, and exercise phases. By combining these streams, BioCoach can access structured biomechanical data unique to each joint, allowing it to provide detailed, anatomy-specific feedback.
Testing BioCoach's Performance
The researchers pitted BioCoach against top video-language AI programs developed by leading researchers and companies in the AI field. BioCoach outperformed its nearest competitor, Stream-VLM, in text quality and judged correctness when responding to videos from the original QEVD dataset. However, the real test came when the programs were graded against the more specific annotations added by the researchers. Here, BioCoach excelled, particularly in biomechanical correctness and detailed, anatomy-specific feedback.
The Future of AI Coaching
The researchers suggest that adding explicit 3D kinematics and biomechanical context can significantly improve the quality and interpretability of real-time exercise feedback without compromising responsiveness. They plan to further enhance BioCoach to estimate joint reaction forces and muscle activation patterns from videos, enabling the detection of slight compensatory movements that could lead to injuries. Ultimately, they envision a system that supports exercise and physical therapy apps, extending the expertise of human coaches and trainers between in-person sessions, and providing users with specific, timely feedback during their independent practice.
This innovative work, supported by the National Science Foundation, showcases the potential of AI to revolutionize the fitness industry, making expert guidance more accessible and helping individuals achieve their fitness goals safely and effectively.
