Machine vision cameras power multi-object tracking system

Competitive sports players may say that practicing fundamentals and physical training represent key elements that determine victory or defeat on the playing field. Understanding team dynamics and the behavior patterns of opponents also can make or break a winning team. Breaking down the specifics and subtleties of play enhances sports coverage and the viewer experience.

The TRACAB optical sports tracking system, developed by ChyronHego (Melville, NY, USA; www.chyronhego.com), serves all these facets of competitive sports by using high-definition (HD) cameras, deep convolutional neural networks (CNN), and cloud-based, data extraction engines and databases to capture and store on-field action for real-time data processing and post-game analysis for coaches, managers, and broadcasters

TRACAB uses distributed camera placements to comprehensively cover all the action during a game, extracting in real time X, Y, and Z positions for each object on the field including players, referees, and the ball, and calculating the center of gravity of all these objects.

The position, speed, and acceleration data captured by the TRACAB system can quickly be used to calculate the workload of each individual player during a match, by providing statistics such as number, distance, and intensity of sprints, and total distance covered. The data can help coaches decide when to make substitutions during the match and prevent players from getting too tired and suffering injury as a result.

ChyronHego introduced the first generation of the system in 2005. Most current deployments use the fourth-generation system, which uses two clusters of three HD cameras, set up between 10 and 20 m of each other in a stereo configuration.

The fourth-gen system is currently installed in more than 300 stadiums and captures footage from 4500 games per year, the European football community, including the German Bundesliga, Spanish La Liga, and Dutch Eredivisie football leagues, as well as UEFA and FIFA tournaments, use TRACAB most widely. The English Premier League and Major League Baseball in the United States have also used the system in their stadiums.

Released in May 2019 and currently installed in 78 stadiums in the German Bundesliga and Spanish La Liga, the TRACAB Gen5 system typically uses six camera clusters, with one or two on each side of the field. In addition to evolving the number of camera units as ChyronHego iterates on the TRACAB system, the company has also pursued the use of more powerful cameras.

The fifth generation of the system uses Genie Nano GigE cameras from Teledyne DALSA (Waterloo, ON, Canada; www.teledynedalsa.com), running at 1936 x 1216 resolution with a frame rate of 25 fps for European television broadcasts and 30 fps for American broadcasts. TRACAB Gen 5 also utilizes a cluster of three Windows 10 workstations from Dell (Round Rock, TX, USA; www.dell.com).

Any camera used in a system like TRACAB, which captures the frenetic action of competitive sports, must be designed to capture and transmit data at fast speeds. Significant distance between cameras and other components may exist, given the size of a sports arena. The Genie Nano GigE can transfer data at rates up to 5 Gbps using Cat5e or Cat6 cabling, for cable lengths up to 100 m, and use fiber optic cable for longer lengths. ChyronHego also chose the camera based on its ability to operate within a wide temperature range and changing light conditions.

FIFA officially approves the use of and certifies the accuracy of nine electronic and performance tracking systems (EPTS). Some use global positioning systems. Others utilize local positioning systems. Two systems, including TRACAB, deploy optical tracking systems. Of the nine systems, by the numbers (bit.ly/VSD_TRCB) TRACAB is clearly the most accurate.

The TRACAB system provides utility for television broadcasters, as well, by enabling 3D recreations of plays viewable from multiple angles and highlighting individual players in the footage. Data extraction modules allow commentators to create analysis graphics like heat maps, pass charts, and move types, and thus illustrate the areas on the field where most of the action takes place. Polish football league Ekstraklasa’s broadcast arm, Ekstraklasa Live Park, in 2017 adopted TRACAB specifically for use in its live coverage.

TRACAB can also improve sound quality during a broadcast by tracking the position of the ball and increasing levels from the cameras closest to the location of the ball, to better capture sound from the most relevant locations on the field. The German Bundesliga deployed automated audio mixing in 2017 and FIFA deployed the system at the 2018 FIFA World Cup in Russia.

ChyronHego is currently developing limb tracking functionality for use in future versions of the system, requiring the incorporation of deep convolutional neural networks. Analysis at the limb level can inform training regimens by calculating physical stress on joints or detecting the direction a player looks and indicate whether the player demonstrates appropriate situational awareness of the location of the ball and other players. Referees might also use limb-tracking data to verify their on-field calls.