New motion capture tech allows your dog to be a video game character

Currently, video gaming is a hobby that you can’t share with your dog. However, that may change soon.

Researchers at the University of Bath in England have developed a motion capture technology that enables you to digitize your dog without the need for complex and expensive motion capture equipment.

Motion capture, without the funny suits

Motion capture technology is widely used in the film and video game industries. Here, actors don suits dotted with white markers—which are then tracked in 3D space by multiple cameras taking footage from multiple angles. This data is then mapped on to a digital character for use in games or films.

In addition, doctors and biomechanical experts have also begun to use motion capture to track the movement of athletes during training, or to monitor patients’ rehabilitation from injuries.

This technology, however, tends to be both bulky and expensive. Even more so, when applying them to animals.

To allow for the motion capture of dogs without the need for bulky and expensive equipment, computer scientists at the University of Bath’s CAMERA motion capture research center have completed all the heavy lifting—capturing the motion of 14 different breeds of dogs.

The dogs were all from the Bath Cats’ and Dogs’ Home (BCDH). Wearing motion capture suits, the dogs were filmed under the supervision of their BCDH handlers while going through a range of movements as part of their enrichment activities.

With the captured data from the dogs, the scientists were able to create a computer model that would accurately predict and replicate poses of dogs filmed even without motion capture suits. Using the model, 3D digital information for new dogs, including their shape and movement, can be captured with just a single RGB-D camera.

“This is the first time RGB-D images have been used to track the motion of dogs using a single camera, which is much more affordable than traditional motion capture systems that require multiple cameras,” said doctorate researcher Sinéad Kearney.

RGB-D cameras work like regular cameras in that they record the red, green and blue color for each pixel in an image. The “D,” however, means that it can also record depth or the distance from the camera of each pixel in the image. 

Avid gamers may already be familiar with RGB-D cameras. Microsoft used the technology for its Kinect, the motion capture camera for the Xbox 360 and Xbox One.

From video games to veterinary medicine

Kearney said that their research can be used by the entertainment industry to produce more authentic movement of dogs, and other virtual animals, for both films and video games. In addition, she also said that it can allow owners to create digital representations of their dogs—something Kearney says “is a lot of fun.”

However, the technology promises to have applications even outside entertainment and may help treat dogs that have suffered physiological injuries.

“This technology allows us to study the movement of animals, which is useful for applications such as detecting lameness in a dog and measuring its recovery over time,” she explained.

“Our research is a step towards building accurate 3D models of animal motion along with technologies that allow us to very easily measure their movement,” added Professor Darren Cosker, director of CAMERA. “This has many exciting applications across a range of areas—from veterinary science to video games.”

Motion capture is already being used to assist in physical therapy for people. For dogs and other animals, however, the need to adapt motion capture suits to various breeds and sizes has made it harder to use.

With this new technology, veterinarians and animal physical therapists won’t have to worry about having the proper motion capture suit for a given animal. (Read: You wouldn’t hit a dog, so why kill one in Minecraft? Why violence against virtual animals is an ethical issue)

More than just dogs

The scientists presented their initial findings at the Computer Vision and Pattern Recognition (CVPR) conference last June. Moving forward, the team is now looking to expand the technology to cover other animals.

Currently, they’ve started testing their technology on computer-generated images of other four-legged animals such as cats, horses, lions and even gorillas, with some promising results.

For the future, the team aims to extend its animal dataset to make the results of the capture more accurate. They also plan to make it available for non-commercial use by other users.