Meta and BMW Seek to Solve In-Car VR

Meta and BMW say they’ve made a breakthrough in their car-based virtual reality project.

The two companies on Wednesday (May 17) announced an update to their partnership, formed in 2021, to explore how augmented reality (AR) and virtual reality (VR) could be integrated into smart vehicles.

“VR headsets are equipped with a number of sensors but moving vehicles pose a tricky challenge, because tracking technology like Oculus Insight use both inertial motion sensors (IMUs) and cameras to precisely estimate the headset’s location and motion,” Meta said in a news release.

These modalities come into conflict in a moving environment, as cameras observe motion relative to the interior of the car, while the IMUs are focused on the “acceleration and rotational velocity relative to the world.”

To solve this problem, Meta worked with BMW to incorporate IMU data from a BMW vehicle’s sensor array in real time into Meta’s Project Aria research glasses’ tracking system, letting the system calculate the glasses’ location relative to the car. 

“That was a huge feat because after transferring the tracking system to a Meta Quest Pro, it allowed us to accurately anchor virtual objects to a moving car using a digital twin of the car,” Meta said. 

The company says it wants to continue working with BMW to leverage machine perception capabilities found in modern cars for additional uses.

“If we get it right, this technology could revolutionize travel in cars, trains, planes and beyond, unlocking new forms of hands-free communication, entertainment and utility,” Meta said.

The announcement comes as companies continue to grapple with how to incorporate VR and HR into their offerings.

As PYMNTS wrote in March, companies including Disney and Microsoft have scaled back their metaverse plans. 

For Disney, that meant shutting down the division responsible for developing metaverse strategies, while Microsoft shifted its focus from its commercial metaverse plans to an “industrial metaverse” strategy.

More recently, Amazon announced it would begin permitting consumers to purchase real-world products from inside virtual spaces.

As PYMNTS reported last week, Amazon Anywhere is an “immersive shopping experience” that lets shoppers browse and purchase physical products from the retail giant from inside of virtual worlds like video games, augmented reality (AR), and mobile apps.

The program debuted within Peridot, an augmented reality game from developer Niantic that lets players control artificial intelligence-powered pets.

“By working with developers like Niantic we are able to showcase new and creative ways for customers to discover relevant products beyond traditional online storefronts, enabling more immersive and engaging experiences both in-and-out of virtual worlds,” said Steve Downer, vice president of consumer electronics at Amazon.