3D Avatar Accurately Recreates Movements of Athletes

John DeWaal    

For athletes, subtle movements can become the difference between winning or losing. Tracking fluid dynamics and body motion helps provide valuable information on improving performance. The process presents several challenges. The shape of an athlete's body and equipment is inherently complex, for instance. The overall challenge of capturing a realistic simulation quickly enough to offer feedback also requires a large amount of processing power.

By combining 3D simulation software and artificial intelligence (AI), human performance optimization company Falcon Pursuit (Portland, OR, USA; https://www.falconpursuit.com/) developed a patented, efficient method for producing full-body, 3D avatars to help push the bounds of human performance.

Creating virtual athletes

Computational fluid dynamics lets users test and experiment the effect of fluid flow over the surface of digital models created by CAD software or 3D scanning systems. CAD software manipulated by a skilled engineer may take days to generate a complex model such as an athlete, which has led to increased use of scanning technologies. 3D scans of complex shapes generate point clouds that engineers use to create highly accurate digital models for use in fluid dynamic software.

Falcon Pursuit 3D simulation software uses point cloud data and a library of high-fidelity avatars to analyze how athletes move in 3D space, with accurate computational air and fluid monitoring. The software includes an AI framework to reduce limitations and time in computational fluid dynamics while maintaining accuracy.

The process begins with a 3D scan used to create a custom 3D model of an athlete. Next, a base avatar (Figure 1) fully rigged with a skeletal structure and muscles “shrink wraps” around the scan of the athlete. Melding specific 3D scan data with the base avatar creates the custom avatar for that athlete.

“Falcon Pursuit software takes the raw data from multiple sensors and processes it with a proprietary algorithm to stitch together overlapping, multiple point clouds into a nicely formed human without holes or gaps,” says Jay White, CTO, Falcon Pursuit. “This may be used for shrink wrapping to a base avatar, forming the final ‘looks like you and moves like you’ Athlete-Avatar.”

Tracking each individual piece of data is not necessary after saving the avatar, and the software removes the point cloud to greatly reduce processing requirements. Additionally, the avatar from the library moves more elegantly than scans or the STL files generated by CAD software. This approach maintains the complex shape and accurately tracks the athlete’s movement while also optimizing the image processing tasks required for real-time feedback on an athlete’s performance. 

Machine Vision Advances Fluid Dynamics

Athlete scanning takes place inside the company’s AVA kiosk (figure 2) and completes in just minutes. Sensors and cameras mounted on robot arms move around the athlete’s body during the scan process. These digital renderings offer precise modeling of the athlete from their form, figure, and 3D motion.

The AVA kiosk uses seven Structure Core sensors from Occipital (San Francisco, CA, USA; https://www.occipital.com/) that each feature stereo infrared cameras, an RGB camera with 1280 × 960 resolution and 160° FOV, a NU3000 multi-core processor from Inuitive (San Jose, CA, USA; https://www.inuitive-tech.com/), and six-axis inertial measurement unit (IMU).

It also employs four UI-5581LE board-level cameras from IDS Imaging Development Systems (Obersulm, Germany; https://www.en.ids-imaging.com/home.html). The cameras feature the 5 MPixel MT9P006STC color CMOS image sensor from ON Semiconductor (Phoenix, AZ, USA; https://www.onsemi.com/) with 2560 × 1920 resolution and a proprietary lens that minimizes “flat view effect” for extremely close subject distance.

The sensors, cameras, and Falcon Pursuit software run on a Neousys Technology (New Taipei City, Taiwan; http://www.neousys-tech.com) NUVO 7000 PC with i7 8700 TE processor from Intel,16 GB of memory, and GTX 730 TI GPU from NVIDIA (Santa Clara, CA, USA; https://www.nvidia.com/en-us/) all of which was custom built by CoastIPC (Hingham, MA, USA; https://www.coastipc.com/).

“The GPUs require quite a bit of power to generate 3D images of the athlete, so we required a PC that can handle the processing requirements,” says Jay White, CTO, Falcon Pursuit. “CoastIPC provided us with a PC that is powerful enough that not only can we generate the 3D images, but we can also manipulate the avatar as often as we need.”

Falcon Pursuit leverages AI to interpolate 2D motion into 3D movement profiles. For example, the software can analyze video of an athlete performing and recreate the same movement in 3D using the athlete’s custom avatar. The software also offers the ability to manipulate the avatar into hypothetical stances or otherwise customize the avatar’s movement to analyze how the changes would affect performance.

Equipment Fittings and Drag Reduction

Drag plays a critical role in an athlete’s performance, especially in sports such as swimming, cycling, racing, and gymnastics. Drag profiles can significantly affect an athlete’s competitive performance. 3D avatars can run through different simulations within Falcon Pursuit’s program in various positions and clothing to test drag profiles. The software also simulates testing in multiple environments—such as underwater settings for swimmers, tunnels for bikers, or windy conditions for runners—to see how performance may improve in different stances and settings.

Coaches looking to create full-scale athletic performance profiles over the course of their athletes’ training can use Falcon Pursuit’s turn-key Avatar Rending Engine (ARE) technology can scan athletes and create avatars on the fly.

At Auburn University, for example, coaches use ARE to monitor the improvement of their football players and gymnasts. The staff uses conventional cameras to capture the athletes’ movements and marry them to their custom avatars. Using these 3D avatar data, the coaches review the techniques of their players over multiple throws, runs, and other activities, pinpointing the shortcomings and strengths of each athlete. The teams’ sports physical therapists, off-season athletic trainers, and other specialized personnel involved in the athletes’ training can then view the data.

A football coach may see that a player puts more weight on their back foot for a long pass, for example. Weight distribution is extremely important for throwing motion, so poor balance and weight distribution results in lower accuracy, limiting the range of a ball’s flight. Avatar analysis can determine precisely how much extra weight is placed on the back foot and what stance alleviates the problem. The player may be more inclined to accept the necessary, targeted training regimens without taking the advice as criticism of their performance, as the targeted training is based on visual data the athlete can clearly see in the simulation.

In the future, Falcon Pursuit believes this accurate computational fluid dynamics model could be used for mechanical, thermodynamic, and hydrodynamic simulations, opening the door to myriad new applications.

Note: The methods described in this article are patented under US Patent Numbers 9,797,802 and 10,648,883.

John DeWaal is the President of CoastIPC.