Augmented reality is nothing new. Google has been working on devices that overlay information and images on top of the real world for years, first with its Google Glass headset, then with Google Tango smartphones. Niantic's Pokemon Go app took the world by storm in 2016 by placing its popular cartoon characters in real life.
But Apple has made it possible to augment reality without investing in pricey new hardware, and its AR apps make Pokemon look like child's play.
We first caught a glimpse of Apple's AR efforts at last June's Worldwide Developers Conference. It was there that the Cupertino company showcased ARKit, a software development kit for developers to make augmented-reality apps for the iPhone and iPad. With ARKit apps now available for everyone to use, here's everything you need to know about Apple's augmented-reality push.
Let's get basic: What is augmented reality?
Devices augment reality by placing virtual objects in your physical world, typically by using your phone's camera. You can interact with those objects by looking at your device's display.
What is ARKit, exactly?
ARKit is what Apple calls its set of software development tools to enable developers to build augmented-reality apps for iOS. Most of us will never actually use ARKit, but we see its results and interact with ARKit apps in iOS 11.
Who can use ARKit apps?
Anyone using an iOS device that runs on Apple's A9, A10 or A11 Bionic processors. That includes the iPhone 6s and 6s Plus, iPhone 7 and iPhone 7 Plus, all iPad Pro models, the 9.7-inch iPad released in 2017, the iPhone 8, 8 Plus and iPhone X. That's millions of devices that are able to use AR apps.
Do I need to buy anything or install anything special to use augmented-reality apps?
All you have to do is have iOS 11 installed. ARKit takes advantage of existing iPhone and iPad hardware, including motion-tracking sensors and camera sensors, to make augmented-reality apps possible.
Why would I want to use AR apps? What are they good for?
AR apps can recognize the spaces around you and place virtual objects on horizontal surfaces. For example, Ikea's iOS app, Ikea Place, allows you to position virtual furniture around your house so you can see each item's scale in relation to your space.
Apple demonstrated the Ikea app to me on an iPad. You simply hold your device up and use the app to drop a piece of furniture, such as a chair, in an empty space. You can easily move it around and even see detail, like the texture of its cloth. I found myself walking around the chair to avoid bumping into it, even though it didn't actually exist in the room I was standing in.
When iOS 11.3 becomes available as an over-the-air update this spring, AR apps will be able to place virtual objects against walls and doors, as well as recognize irregularly shaped objects in the real world. The software upgrade will improve the resolution of the camera view in AR apps by 50 percent and add support for auto-focus when viewing augmented objects through your phone's camera.
What about games?
Of course, one of the most fun ways to take advantage of augmented reality is gaming. We saw an early Star Wars-themed game of chess that projects a holographic Jabba the Hutt above a virtual chess board on a table in front of you. Jabba watches your pieces duke it out with his, roaring all the while.
Other games include a "Walking Dead" app from AMC called Our World that hides characters in your augmented world and has them pop out at you. I'm sure some people will find that amusing. (I will not be playing this.)
You can check out more of our favorite ARKit apps and demos here, and follow the @MadeWithARKit Twitter account to see new submissions from developers.
How are those apps different from something like Pokemon Go?
Pokemon Go works by overlaying two-dimensional images on top of real, three-dimensional scenes. ARKit apps put three-dimensional images in your world using what's called visual inertial odometry, which takes advantage of your device's sensors to track your world and sense your device's orientation and position relative to the scene you're looking at.
Your phone or tablet's camera can detect planes and position three-dimensional images on top of them so they look natural. The camera sensors even help apps orient lighting, so augmented objects will cast what appears to be real shadows.
How can I tell which apps have AR features?
The App Store offers a curated section to highlight the best AR experiences. Just tap on Apps and scroll down to Top Categories. AR Apps is at the top.
Does Google offer anything similar for Android phones?
Google offers a ARCore software development kit, which is similar to ARKit in that it adds augmented-reality functionality to existing Android phones. ARCore apps run on Google's Pixel, Pixel XL, Pixel 2, Pixel 2 XL and Samsung's Galaxy S8, and will come to millions more Android devices running Android 7.0 Nougat when it emerges from developer preview. You can see what cool things developers are doing with ARCore here.
