Nvidia has unveiled some really clever new augmented reality visualization technology that may be key to creating glasses that can actually match human’s visual acuity.
In a blog post about all the cool stuff that the company introduced at Siggraph this year, the company announced two new AR devices: the Prescription AR glasses and the Foveated AR headset.
Right now, if you use glasses, using AR or VR displays is either uncomfortable or impossible. While some headsets allow the user to wear glasses, the result is too cumbersome to be comfortable, especially for a long time. If you leave your prescription glasses out, you will not be able to enjoy your experience as much as someone with 20/20 eyesight.
Nvidia claims that 60% of the global population uses glasses, so they have been working to solve this with a new technology that combines an micro-OLED display and a special 5-millimeter thick prescription lenses tailored to the wearer.
In the Prescription AR glasses — which won Siggraph’s Best of Show Emerging Technology award — the digital image goes through a beam shaping lens. The image then goes through a prism that sends it down through half-mirrored path inside the transparent lens until it reaches a free-form image combiner that sends the digital image into the retina mixed with real world vision, corrected according to whatever problem the user has: myopia, astigmatism, or hyperopia. The result is a crystal clear image in which the digital image appears at a fixed focal distance.
According to Nvidia, the Prescription AR glasses have a larger field of view than current AR devices, but it hasn’t disclosed the actual figures.
While it’s cool that eyeglass users such as myself get to use these devices without having to wear two things on our faces, to me the most interesting development is Nvidia's Foveated AR headset.
It works on the principle that eyes have a central part — called the fovea — that has a very high concentration of cones right in the center of the macula, in the retina. The fovea only covers 1% of the retina but uses 50% of the brain’s visual cortex. It’s basically the highest resolution part of our eyes — but it only covers two degrees of our visual field.
The rest of our visual field greatly decreases in resolution as it you go out of that central part. Our peripheral vision is low-res — which is why your eyes are constantly moving on their own, especially when they have to focus on an object that is larger than the field of vision covered by your fovea. You can notice this while reading these lines; your peripheral vision is likely blurred in comparison with the sharpness of these words.
So Nvidia figured out that you don’t really need to render everything in high definition, only the part that the fovea see. The problem is that the eyes are constantly moving precisely because of this biological architecture.
The Foveated AR headset solves this using various elements. First, the system uses a micro-OLED high resolution display to project an image that matches your fovea’s perception abilities right into that part of your retina. A secondary low res projector renders a lower resolution version of the digital image for the rest of your field of view.
The system needs to move at the same time of your eyes so the high resolution part is always focused on the fovea. For that it uses a motor that quickly displaces the visual assembly at the same time as your eye moves, always zeroing on your fovea.
According to Nvidia's research paper, the result of this system is an image that closely matches your visual perception abilities. And since the computer doesn’t have to generate a high resolution image for your peripheral field of view, the processing power, bandwidth, and memory requirements are greatly reduced with this method.
Nvidia also says that the diagonal field of view with this system is over 100 degrees per eye, which is impressive compared to the best available AR headset: Hololens 2, which offers 52 degrees. It’s still not up to par to the human eye, but it’s getting there. For comparison, each human eye has a horizontal FOV of 135º and vertical FOV of 180º. The Nvidia Foveated AR headset achieves 85º by 78º and the Hololens 2 is 43º by 29º.
We don’t know if Nvidia will finally release this technology into a product. But it’s good to know that one of the most powerful graphics houses in the world is putting some money into pushing the augmented reality envelope in such a dramatic way.
