Conception and Development of Posture-Aware Language Models for Automatic Exercise Recognition

Master thesis

Motivation

Traditional approaches to exercise recognition involve training specific classifiers for each exercise, typically utilizing sensor data such as accelerometer and positional data. In addition to motion capture data, one could also incorporate textual descriptions of a posture (e.g., “starting from an upright position, extend both arms forward, then move your left leg backwards”). By establishing the connection between textual descriptions of postures and motion capture data, we can ultimately determine the accuracy of an entire exercise execution.

Tasks

In this thesis, you should explore approaches for full-body pose recognition using a combination of motion capture data and textual descriptions. To achieve this, a comprehensive approach involves the utilization of a full-body motion capture suit equipped with 14 inertial measurement units. This facilitates the collection of sensor data across various physical activities. Subsequently, machine learning models, including those based on Transformers and incorporating Cross-attention mechanisms, will be trained to grasp the relationship between the text description of a pose and its corresponding accelerometer or positional data.

The thesis can be written in English or german.

Requirements

• Experience with transformers/LSTMs

• Good python skills