Apple’s AR glasses may be dead, but the company is reportedly working to release a mixed reality headset in 2020 with 8K displays for each eye and a set of cameras and sensors to track both the environment and the user’s face. The project codename: T288.
Mixed reality headsets are basically straight VR headsets that are equipped with cameras to capture the reality around the user. Rather than overlaying 3D computer graphics on your actual view of the world — like with the Microsoft Hololens — a mixed reality set takes the video capture with the camera and the synthetic image and merges it into a video.
A CNET story claims that a new Apple patent filed with the United States Patent and Trademark Office is proof of their previous April 2018 report. The patent — published on July 18, 2019 after a March 2019 filing — describes a headset with one display for each eye, cameras, and sensors to capture the user's face. This will not only allow the headset to mix reality and 3D into one, but will also track the user's expressions to render its avatar in cyberspace.
Apple has had a long list of patents and company acquisitions that clearly show that the Cupertino company is working on AR glasses.
Tim Cook has said in the past that he is a fan of AR, thinking that VR is too closed and isolating, while AR is about connecting with others and the real world. As a result of this AR strategy, Apple started to buy companies related to augmented reality as far back as 2015, when it acquired Metaio and the software code that formed the basis for Apple ARKit, the company’s augmented reality development kit for iOS.
In 2017 alone, Apple bought InVisage Tecnologies (an American quantum dot-based image sensor manufacturer), Regain (a French computer vision company), Vrvana (a Canadian manufacturer of augmented reality head-mounted displays), and SensoMotoric Instruments (which makes eye tracking hardware and software). Most recently, Apple bought Akonia Holographics and its 200 AR patents.
Last July, Apple added another augmented reality patent in that long patent list. Titled “Augmented Reality Device to Warp Images”, the patent published by the USPTO describes AR glasses designed to help people with visual impairment by capturing the view of the user and “warping” part of the reality to make it bigger. This is something that our brain does naturally when we need to focus on something fast moving, like a ball coming towards you: the brain automagically oversizes the ball so you can catch it better, one of the many tricks that evolution — and our need for survival — has put at our disposal. Apple’s system works in a similar way, but digitally.
All these technologies could be applied to this rumored AR/VR mixed reality set. It is not the pure AR that Cook was evangelizing but — given the physical limitations of current optical and processing technologies that make the latter impossible at this point — the mixed reality headset is the next best thing Apple can do. Perhaps that’s what we will get in 2020, after all.
