Computer vision in sports: people train and compete. Machines watch and help

At the Tokyo 2020 Olympic Games, Intel’s 3D Athlete Tracking turned ordinary broadcast footage of sprint races into 3D skeletons for replays. Four years later in Paris, the host broadcaster’s replays rebuilt athletes in 3D from multiple camera angles, and Omega used computer vision to measure sports such as beach volleyball and diving. Artificial intelligence and computer vision systems in sports are no longer a high-tech novelty but an everyday reality. People train, challenge, and watch others compete — and hundreds of tech companies are helping to make it safer and more efficient.

Despite the current boom in machine intelligence and computer vision technologies, the first commercial developments in this field are now more than 30 years old.

Back in the mid-1990s, Ramm Mylvaganam, who had quit his job at Mars, decided to devote himself to the two things he loved most: data and sport. After talking to managers of British football clubs, he decided to build a system that would help evaluate the quality of play and players' performance analytically rather than subjectively — and created Prozone.

To do this, a major technological leap had to be made because the existing technologies of tracking with sensors or GPS achieved the accuracy of positioning a ball only to a few meters — and Ramm Mylvaganam needed centimeters. Or at least tens of centimeters.

He invested millions of pounds in developing a camera system and machine vision technology to get real-time data on the position of players and the ball. This data has helped to numerically measure the player's performance, helping coaches tailor training and game plans for each individual player.

Prozone became one of the best-known names in football analysis. It later joined forces with Amisco and in 2015 was acquired by STATS, today’s Stats Perform.

Today, there are hundreds of companies using computer vision for sports analytics and artificial intelligence in the sports market, large and small. Simple applications help count push-ups and squats. And the most complex systems — to measure milliseconds and millimeters at the Olympic Games. Let's take a look at some key examples of how AI is being used in sports.

AI and decision making

Computer vision and AI are helping amateur and professional athletes train more efficiently. For example, Google DeepMind and Liverpool FC built TacticAI, a research system trained on 7,176 Premier League corner kicks. It predicts which player is likely to receive the ball and suggests how to reposition players; in a blind test, Liverpool’s experts preferred its suggestions 90% of the time. On the tracking side, Stats Perform’s SportVU 2.0 collects player positions from high-speed cameras.

Computer vision can also follow individual limbs, turning video into joint angles and movement patterns that coaches and medical staff can review over time. In American football, the NFL and AWS built the Digital Athlete, which combines tracking, video, equipment and weather data to simulate plays and study injury risk. Separately, the league’s Big Data Bowl invites analysts to work with its player-tracking data.

There are also projects that combine AI for decision making and proprietary development in hardware. For example, Smart Coach creates smart trainers. Artificial intelligence in sports collects personalised training data and creates individual programs and plans — the solution is positioned for the professional athlete market.

At the same time, artificial intelligence in sports is available to everyone. For example, the HomeCourt service offers a free application for basketball training. The app uses the smartphone camera to recognize players and shots on the court and keeps statistics on the number of shots and speed of movements. For experienced players, there is a subscription to online training and synchronization with wearable devices.

AI and sports refereeing

Hawk-Eye, part of Sony since 2011, has been a fixture of elite sport for years: it has provided goal-line technology in the Premier League since 2013 and now works in 25 sports in more than 100 countries. At the 2014 FIFA World Cup, goal-line decisions came from a different system, GoalControl-4D, with 14 cameras per stadium. Automated help with offside came later: semi-automated offside technology made its World Cup debut in Qatar in 2022, combining 12 tracking cameras with a sensor inside the Al Rihla match ball. Camera-based systems need no chip in the ball, unlike earlier approaches such as GoalRef, but they are not cheap: goal-line technology for Premier League grounds was reported at around £250,000 per stadium.

The involvement of AI in judging is causing ethical controversy, among other things. For example, activists and researchers advocate for transparency in the operation of AI judging algorithms. Otherwise, the tuning of algorithms by bookmakers and clubs could increase the risk of match-fixing. After all, in the minds of many viewers, “artificial intelligence doesn't make mistakes”.

Today, video tracking technologies are used in all types of sports: from snow jumping to motorcycle racing. And the number of companies offering such solutions is in the hundreds.

Spectacle

Even Amazon is involved in technologies to enrich the viewer's experience — together with Formula 1, the company has introduced a technology that helps combine AI capabilities in data processing and augmented reality in race broadcasts. Formula 1 teams collect a lot of data from sensors installed in their cars, which is processed by AI and visualized during broadcasts.

This way, race fans get more insights into how the race is going and what condition their favorite teams and cars are in. This is an important market — after all, Formula 1 earns hundreds of millions of dollars from race broadcasts. AI gives this market significant growth opportunities.

Stats Perform focuses on sports data and how it is presented. The company covers more than 500,000 matches a year across 3,900+ competitions and 20+ sports, holds more than 7 petabytes of data, and runs over 140 AI models to process and visualize it.

Video broadcasts can be enriched not only with data but with ads too. For example, Mirriad applies real-time virtual advertisements to the surfaces of stands or car bodies during broadcasts of races and competitions. This allows millions of viewers to receive relevant and noticeable advertising and sports teams and clubs to earn extra money by moving away from complex and boring traditional advertising models.

At the same time, cameras and machine vision systems can be aimed not only at the field but also at the stands. For example, Ipsotek, now part of Eviden, makes VISuite, which helps count spectators and spot potentially dangerous situations such as fan clashes. Thanks to machine vision, security can arrive at the scene within seconds. In 2021 Major League Soccer’s Columbus Crew introduced opt-in facial-recognition entry, letting fans walk into the stadium without a paper or phone ticket.