Overview

Due to contractual obligations, only a high-level overview of the following three interrelated projects is provided.

Table of Contents

  1. Exercise Repetition Counting
  2. Human Activity Recognition (HAR) with Deep Learning
  3. Fine-Grained Human Motion Comparison

1. Exercise Repetition Counting

I developed a precision exercise repetition counter specialized for gym workouts. The system uses four IMU (Inertial Measurement Unit) sensors attached to different parts of the body, depending on the exercise. Its parameters can be optimized via Bayesian optimization with Gaussian processes, allowing fine-tuning for specific exercises or individual users when sufficient data is available. Notably, it works “out of the box” without custom training data. See Figure 1 for a demonstration. While large-scale validation is pending, the counter has demonstrated near-perfect accuracy on limited test data.

Figure 1. Exercise repetition counting (and recognition). Counts are shown in the small text at the top left corner, along with the time information of peaks recognized as exercise repetitions.

2. Human Activity Recognition (HAR) with Deep Learning

Building on the repetition counter, the system precisely marks the start and end of each repetition. Once isolated, the IMU sensor data is preprocessed, using methods such as SLERP (spherical linear interpolation) to ensure each repetition’s data is consistently sized. The standardized data is then fed into a custom deep neural network that classifies each repetition in real time on a mobile device, identifying the specific exercise performed. Although comprehensive evaluations require larger datasets, tests of an 11-class classifier have achieved near-perfect accuracy on the available data.

Figure 2. Visualization of exercise data using t-SNE.
Exercise Recognition
Figure 3. Demonstration of exercise recognition. The boxes at the bottom indicate the type of exercise identified by the deep-learning classifier (a zoomed-in screenshot from the animation in Figure 1).

3. Fine-Grained Human Motion Comparison

This project compares individual time points from a user’s exercise repetition against a reference sequence, be it an expert’s ideal form or the user’s own past performance, to assess technique and track improvements over time. To accomplish this, I customized Dynamic Time Warping (DTW) for more precise alignment of the IMU data across repetitions. A key challenge was devising an effective method to compare body-position data so that the motion matching remains both accurate and robust. See Figure 4 for a demonstration.

Figure 4. Demonstration of matching a single exercise repetition (green) to motion time-series data (grey). The exercise in the animation is deadlift.

For more information on Dynamic Time Warping, please visit the Dynamic Time Warping — Wikipedia page.