During the pandemic, everyone has got a lot of mileage out of changing Zoom backgrounds to liven up dull meetings. But it looks like Apple is assessing the same kind of technology for its upcoming AR specs, the Apple Glass.
First spotted by Patently Apple, the patent describes how a headset could perform chroma keying – which is replacing a solid color background with something else. This may not sound too taxing, but the trick is to make it a hardware solution on the headset, because even slight delays could otherwise prompt motion sickness. Add to that the fact that a headset is constantly moving unlike a Zoom-enabled laptop, and you can see why this could prove quite a challenge.
- Apple Glass: Release date, price, features and leaks
- Check out all of the latest iPhone 12 news
- Just in: Google Pixel Watch design reveals a gorgeous Apple Watch 6 killer
First filed back in February, but only published publicly this week, Apple’s patent is about creating a low latency 3D stereoscopic background for Apple AR. What this comes down to is “a wearable immersive head-mounted display (HMD) doing real-time color-keying using embedded stereo RGB cameras to capture the environment and a display system to show real-world images augmented with virtual content.”
It seems like Apple has had some success with this challenge. “The system runs at a high frame rate (typically 75 fps or more) and it achieves a latency of less than one frame by capturing images and formatting them for display in the HMD itself," the patent says.
How has Apple done this? By offloading the processing to the headset, which can, according to the patent, “format camera images, detect the selected color range and make a composite with the virtual content. “ Clever stuff.
Along with the fact that people tend to move their heads when wearing a headset, there’s another issue Apple has to contend with: all of this has to be done twice, once for each eye. “The system achieves mixed reality by having a user look at a display through wide angle lenses (FOV is typically 110 degree or more)," the patent reads. "Two cameras located behind the HMD display capture the environment from viewpoints located a few centimetres in front of each eye."
All very nice, but what’s the point? As you might expect, the patent doesn’t go into massive amounts of detail here, keeping things vague: “games, videos and virtual objects.”
We won’t have concrete answers any time soon, either. People in the know differ about when we’ll see Apple Glass for the first time, but the earliest estimate is March next year, with well connected analyst Ming-Chi Kuo saying 2022 at the earliest. A report from last year suggested 2023 was more likely.
It’s also worth remembering that Apple patents an awful lot of things – over 2,500 bits and pieces in the last year alone – and not everything gets used. So while you may well be applying funny backgrounds to your Apple Glass in 2023, it’s equally possible that that’s another feature left on the cutting room floor.
