AI supported jump recognition and analysis

This post is a short overview of an Abto Software education and e-learning project.

Our client is an Ireland-based organization, which contacted our engineers to provide technical consulting, solution discovery and design, and other accompanying services to deliver an enterprise-level school platform. The project was focused around adopting jump recognition and analysis by implementing artificial intelligence.

Having entered into cooperation, our company successfully integrated artificial intelligence to implement advanced camera-based movement recognition to extend an already existing enterprise-degree school portal to facilitate public health across local educational facilities.

Sensorless human activity recognition: Key goals

Our client was lacking technical expertise to leverage computer technology, in particular artificial intelligence. After conducting thorough investigation, our engineers have performed data collection and pre-processing, model selection and training, testing, validation, and deployment.

Our team seamlessly adapted complex algorithms that can:

• Accurately recognize physical activity in teal-time
• Promptly evaluate and measure performed exercises
• Automatically compare assessed exercises with the reference pattern already pre-defined
• And return precise estimates

Sensorless human activity recognition: Our contribution

Abto Software was involved to implement cutting-edge technology, in particular pose detection and analysis, and provide:

• Business consulting
• Solution discovery and design
• PoC development
• Cross-platform development
• Integration planning
• Solution deployment

Our company, having required domain knowledge and experience, also suggests implementing guides to help the users:

• Position themselves in an appropriate distance to the smartphone/tablet camera
• Detect obstacles that might potentially deteriorate an accurate landmark detection

Tech stack: Python, MediaPose, SciPy, OpenCV, Pandas, Numpy, and PyTorch.

The product: Quick overview, concept, functionality

In the first prototype we introduced, the model was working on ready-made, already captured video material. After the solution’s implementation, the algorithms are working with live video streams automatically derived from PC’s, smartphones, tablets, and laptops.

The platform was introduced to improve public health across local educational facilities, uppermost schools. Our concept is designed for straightforward jump detection and analysis, but might be extended to empower involved workers in assigning and monitoring other exercises, including squatting, stretching, yoga.

The concept is simple:

• The student is shown an easy-to-follow reference pattern of a specific exercise
• The student then performs the exercise (for example, jumps up, balance exercises, or others)
• The system quickly compares the execution with the reference pattern and evaluates the performance
• The student or the responsible teacher can review the results to make further adjustments

The solution might benefit:

• National authorities that aim to improve public health
• Educational institutions that aim to improve student wellbeing and motivation
• Physical therapists, personal trainers, sports teams, and private fitness centers
• Individual users, both children and adults

The project: Main challenges worth mentioning

The project’s main challenges were focused around investigating and implementing AI based pose detection.

Having knowledge and experience in applying computer technology, we integrated AI enabled pose estimation by handling:

• The creation of a reference pattern to compare and evaluate performed exercises
• The training of the selected model to assess and analyze captured activity
• The stabilization of the movement recognition
• The development of techniques to compare and evaluate human proportions (height, weight)

Final words

Abto Software has covered business consulting, solution discovery and design, PoC development, and smooth integration planning to incorporate jump recognition and analysis into enterprise-level school platforms, eventually facilitating public health across local educational facilities.

We can also provide:

• .NET development
• ASP.NET development
• Web development
• Mobile development
• Cloud services
• Custom product software development