I can’t believe it’s been more than a decade since Samsung introduced what’s arguably its most gimmicky feature in a phone. I’m referring to the Air Gestures of the Galaxy S4, which were simply too ahead for their time. Remember them? Probably not.
Despite the novelty of waving my hand to scroll up and down web pages, or scrolling through the picture gallery, Air Gestures were largely forgettable — and probably why the feature slowly receded in subsequent editing of Samsung's phones. It simply wasn’t as intuitive as I thought it would be, especially when I could just do the old-fashioned thing of using my fingers to scroll.
Yet, I feel like Aor Gestures are long overdue for an upgrade, and the feature could be the key to the Galaxy S24 Ultra’s salvation. That’s because of the S Pen, which I suspect will accompany the upcoming flagship.
If there’s another reason why the Galaxy S24 Ultra is making a strong case for one of the best phones in 2024, bringing back Samsung’s Air Gestures could be it. Of course, that feature needs a worthwhile upgrade.
Make the S Pen as a mouse
Interestingly enough, Air Gestures are still around, but not to the same degree as when when they first debuted. The Galaxy S23 Ultra uses a variation of Air Gestures with the S Pen, called Air Actions, that lets users perform navigational functions — such as home, back, and recent apps. And fun fact, you can also use the S Pen in apps like Chrome to scroll up/down pages, which you can see in the animation above.
But there’s an opportunity to improve this in a meaningful way, especially if Samsung DeX is going to have a critical role in how it could be used. Wouldn’t it be helpful if the S Pen acted very much like a mouse — one that you can use in the air, rather than on a flat surface. It would require adding in a gyroscope and accelerometer into the S Pen, which could allow the stylus to precisely navigate around DeX’s interface.
In the current iteration of Samsung DeX on the Galaxy S23 Ultra, I can set the phone’s display to act as a virtual trackpad to help me move the mouse cursor. I think it’d be neat to achieve the same function by waving the S Pen in the air, while the physical button on there could act as the clicking function.
Pinch zooming made easy
From the looks of it, the scrolling function of the S Pen with the Galaxy S23 Ultra is shared among the phone’s many apps, such as Chrome and the Gallery app. What’s missing is the ability to perform pinch gestures to zoom in and out.
There’s a simple way for Samsung to achieve this with the Galaxy S24 Ultra, but the problem lies in distinguishing these Air Gestures from one another. Rather than interfering with the current up, down, left, and right gestures of the S Pen, Samsung could enable pinch zooming with push in/out gestures while holding down the S Pen’s button.
With an accelerometer and gyroscope in the S Pen, pinch zooming via Air Gestures could be extremely accurate, and combined with the other navigational Air Gestures, the S Pen could be the navigational tool I’ve been waiting for better efficiency. Like how neat would it be to easily scroll through websites and zooming into things with intuitive gestures?
AI could boost Air Gestures with our hands
If these possibilities sound too good to be true, Samsung could take us all by surprise and bring back Air Gestures with our hands. It’s a tall order to achieve, but with all the talk about marketing an ‘AI Phone’ that will be the showcase for future phones released next year, I think it’s in the realm of possibilities.
There’s a reason why Steve Jobs made a case for the iPhone to use a capacitive touchscreen instead of the resistive ones that were commonly used in smartphones at the time. Our hands and fingers are tools of their own, so it makes sense to rely on them for Air Gestures.
But the biggest obstacle would be a way to perform them intuitively. The last thing I want to see happen with Air Gestures possibly on the Galaxy S24 series are unresponsive gestures. That’s why it would require a balancing act between the software, processor, cameras, and other sensors to make this all happen.
If you’re to look at Face ID on the iPhone, there’s a lot going on for it to accurately detect faces — so a similar implementation could make Air Gestures more reliable on the Galaxy S24 Ultra. For example, one component of Face ID is a dot projector that projects these tiny dots onto someone’s face, which could be leveraged in a way to monitor the movements of our hands in space with the Galaxy S24 Ultra.
Like I said, it’s a process that may sound easy, but it’s actually complex when you break down what needs to happen. I would imagine that on-board artificial intelligence would also improve the reliability of Air Gestures by monitoring the front-facing camera — in addition to the data that would be acquired by other sensors.
Even though Samsung won’t have a new mixed reality headset to compete against Apple’s upcoming Vision Pro, I think the Galaxy S24 Ultra could be a teaser preview of what’s to come if and when Samsung announces its VR headset rival. I’m not saying that the Galaxy S24 Ultra would have direct involvement with Samsung’s headset, but it could be the test ground for new experiences.
More from Tom's Guide
