What Is Skeletal Tracking?
Skeletal tracking is a computer vision technology that detects and maps the human body's key joints and limbs in real-time. Think of it as creating a digital skeleton overlay on your body movements, tracking everything from your head position to your ankle angles.
How Morld Uses MLKit for Pose Detection
Morld leverages Google's MLKit with AccuratePoseDetectorOptions for maximum precision. The system processes each video frame through a deep neural network that identifies 33 body landmarks with sub-pixel accuracy.
- Head landmarks: Nose, ears, eyes — for tracking head alignment
- Upper body: Shoulders, elbows, wrists — critical for arm exercises
- Core: Hips — the center of most movements
- Lower body: Knees, ankles — essential for squats and lunges
From 2D to Pseudo-3D
While a phone camera captures 2D images, Morld's algorithms create a pseudo-3D representation using:
- Perspective foreshortening: Estimating depth from limb proportions
- Z-ordering: Determining which body parts are in front or behind
- Depth-based shading: Visual depth cues for better understanding
Our skeletal tracking system processes frames with just 25ms tolerance, ensuring smooth real-time analysis without lag.
The Scoring System
Each detected pose is scored using a weighted cosine similarity algorithm. Limb joints (arms and legs) receive higher weights than torso or head, because proper limb positioning is most critical for exercise effectiveness and injury prevention.
Joint Weight Distribution
Our scoring prioritizes joints based on their importance to exercise form: limbs are weighted highest, followed by the torso, with the head receiving the lowest weight. This ensures the scoring focuses on what matters most for your workout quality.
Why Accuracy Matters
Even small deviations in form can compound over hundreds of repetitions, leading to muscle imbalances or joint stress. Morld's skeletal tracking catches these subtle issues before they become problems, helping you train safely and effectively.
