Galaxy S9 Bixby Guide
Despite getting off to a slow start a year ago when it debuted alongside the Galaxy S8, Samsung's Bixby virtual assistant has made a lot of progress. With the arrival of the Galaxy S9, Bixby is expanding what it can see and how it can use that information to help you.
Thanks to improvements introduced with Samsung's latest phones, all you need to do is show or tell Bixby what you want it to help with. Still, even with the S9's promised improvements, Bixby's abilities and vocabulary remain limited, especially when stacked up against the more full-featured Google Assistant.
MORE: Get Started With Galaxy S9: What Features to Enable
The main way to use Bixby is to press its dedicated button on the left of the handset and tell it what you're interested in. If you want use the service's improved integration with the S9's camera, start at the Camera app, and then tap the Bixby Vision icon that looks like an eye. Once it's open, you'll find eight Bixby Vision modes, ranging from food to makeup, along the bottom, under the camera's preview screen.
Here's a closer look at what Bixby can do for you on the Galaxy S9, along with what you can do to get started with Samsung's assistant. —Brian Nadel
Setting up Bixby
As with Google Assistant and Apple's Siri, the bulk of Bixby's tasks can be started by giving it a command. You can also summon Bixby with a push of the dedicated button on the side of the Galaxy S9. But before you can do anything, you need to register your Galaxy phone with the artificial-intelligence platform. This way, Bixby can learn from your activities, improve its cognition and, hopefully, even start to anticipate your desires.
Your first step with a fresh phone is to tell Bixby your preferred language. The service speaks only three: English, Korean and Mandarin. That's a fraction of the more than 20 languages Siri understands. Samsung is working on catching up.
After a little training to ensure that Bixby recognizes your voice, you're set. The assistant responds well to canned questions like, "Hey, Bixby, show me my missed messages," and "Bixby, set an alarm for 6:30."
The key is to stick to Bixby's thousands of established commands. These are increasing steadily, but you need to stay on script. I found Bixby to be less effective when I tried commands like, "I want to see my emails" or "where am I?"
What you can do with Bixby
Once you get Bixby up and running, it's time to see what Samsung's assistant can do. Because it's built specifically for Samsung devices, Bixby can handle commands that let you control your phone, without making you endlessly tap at the screen. You can ask Bixby to add events to your calendar, launch the camera, turn on an alarm, show notifications or interact with a variety of Samsung's built-in apps. There's even voice functionality with third-party apps like Instagram, Twitter and assorted Google offerings.
You can also use Bixby as a search tool. The service worked well for asking for this year's Oscar winners. It couldn't, however, show me chain saws for cleaning up after a recent storm; instead, it thought I wanted to change the font size. Here, Google Assistant, with its access to Google's knowledge graph, still has a big edge.
Bixby Vision
Bixby included some visual features last year. For instance, you could point your Galaxy S8's camera at a book, and the Bixby Vision feature would bring up the price of the book in near real time.
Bixby Vision is taking a big step forward in the Galaxy S9, as Samsung has increased the number of ways the assistant's AI engine can work with the phone's camera.
One key addition is the ability to translate street signs, menus and other things in 53 languages. All you do is open the Camera app and press the on-screen Bixby Vision logo that looks like a stylized eye. Aim the lens at what you want Bixby to translate, and you're off.
While it's not quite Star Trek's universal translator, Bixby Vision worked well with a Chinese restaurant's menu to help me avoid eating drunken shrimp. But it couldn't handle the image of a simple French sign reading "Rue de Londres" ("London Street"), which it interpreted as "Aloha Beach." In fact, Bixby tried to sell me lingerie at the same time.
Here's a rundown of other ways Bixby Vision works with the S9's camera.
Scan QR codes: I've used several QR-code readers on my phones over the years and haven't found one that works with everything. Bixby takes a big step toward that goal. Start by swiping through Bixby Vision's modes all the way to the right, until you get to the one that looks like a QR code. When the QR code is centered and the right size, Bixby interprets it and sends you to a website.
In addition to delivering warranty information and manuals, Bixby handled a tough QR task that stymies other QR readers: the validation code on HP toner. Bixby got it on the first try and took me to the correct website.
Become a wine snob: Oenophiles of the world, unite! Bixby can help you imbibe the best wine. Just swipe to the Bixby Vision wineglass icon, and center the bottle's label in the screen's rectangle. (It's awkward to balance the wine bottle, so you might want to put it on a table.) A rating of the wine's vintage shows up, and you can tap to get more details, such as what foods it pairs with.
Go shopping: Bixby has a Shopping mode, designated by a supermarket-basket icon. And it's hit-or-miss when it comes to correctly identifying what you're (literally) looking for.
Bixby worked well with cans of soup and books but utterly failed when it came to a bottle of beer, offering to let me buy shoelaces instead.
Check out your makeup: The Bixby Makeup mode is probably the best-executed of Bixby's effects and can actually help people decide what makeup products to get.
In association with Sephora, Bixby can preview a slew of makeup products, from eyeliner to lipstick, on a live selfie. Just pick the effect from the list below the image, and the actual products used will be at the top. If it's not a pretty picture, tap the circular arrow to start with a fresh face.
Turn images into text: If you have a paper document, Bixby can convert it to text with reasonably good fidelity. Line up the item on the camera's preview, press the Bixby Vision Text mode that has a circular logo underneath with the "T," and Bixby is off rendering it as editable text.
The feature works better on printed material than on handwritten notes, as you might expect, but it's still impressive. Bixby correctly translated a letter and a subscription warning into text.
Count your calories: If you move the Bixby Vision mode to the steaming-bowl logo, Samsung's assistant can identify food for you and give you a calorie count.
The feature worked well for a banana, a hamburger and even a plate of vegetable biryani. On the downside, while Bixby correctly registered that I was eating sushi for lunch, it couldn't identify any of the individual rolls.
Identify objects: Bixby has one other skill that could come in handy once it becomes a bit smarter. You can snap a photo of something — anything, really — and Bixby's cloud intelligence tries to match it with other known objects.
The trouble is, at this stage, you don't always get the right result with object recognition. Bixby correctly identified a thermometer and my pet tortoise but was off the mark with a flashlight (Bixby guessed pen) and flash drive (Bixby thought it was a wallet). Bixby's object recognition is supposed to automatically improve over time as more people use it, though there's no way to correct Bixby.
