Samsung’s Bixby assistant: 10 things worth trying

Samsung’s Bixby gets a bad rap sometimes, and not always undeservedly. It isn’t as knowledgeable as the Google Assistant or as platform-agnostic as Amazon’s Alexa. In fairness, however, it has come a long way since its debut on the Galaxy S8 and S8 Plus in 2017.

Bixby’s available in more places (over 200 countries) and on far more devices than it used to be — like Samsung’s Family Hub 2.0 refrigerators and the Seoul company’s latest-gen Smart TV lineup. (Early next year, the first-ever smart speaker with Bixby built in — the Galaxy Home — will join the club.) On the features front, Bixby has learned to speak new languages, like German, French, Italian, and Spanish. Moreover, thanks to a newly released developer toolkit (Bixby Developer Studio) and a digital storefront (Bixby Marketplace), it supports a greater number of third-party apps and services than ever before.

But what can Bixby do, exactly? An overwhelming number of tasks and chores, as it turns out — Samsung claims it recognizes over 3,000 commands. To help simplify things, we’ve put together a quick starter guide for Bixby. It’s not exhaustive, but with any luck, it’ll help you get a handle on the do-it-all assistant’s basics in no time.

5 things to try with Bixby Voice

For all intents and purposes, Bixby Voice is Samsung’s answer to Alexa, Cortana, and Apple’s Siri. That is to say: It’s a (mostly) intelligent assistant that responds to commands

It’s easy to get Bixby Voice up and running on the Galaxy S8 and S8+, Galaxy S9 and S9+, and Galaxy Note9. Simply press and hold the dedicated Bixby button on the left side (below the volume rocker) for a few seconds and follow the on-screen prompts. Alternatively, head over to Bixby Home — the left-most dashboard on the phones’ home screens — by quickly touching the Bixby button and launching the More Options menu, the latter of which is designated by a three-dot icon in the upper right-hand corner. Then, touch Settings and toggle on the Bixby Voice slider.

In either case, you’ll have to sign up for a Samsung account if you haven’t already. And you’ll be asked to enroll in Samsung Rewards, a program that offers digital “experience points” — which can later be traded in for gift cards and coupons — in exchange for using Bixby to accomplish certain tasks, such as making a call, sending a message, or changing a setting.

If hands-free commands are more your style, pull up Bixby Home, open the More Options menu, touch Settings > Voice wake-up, and toggle on the top-most slider. Then complete the instructions, after which you’ll be offered the option of recording or re-recording, listening to, or deleting the wake-up command.

Getting Bixby’s attention on non-phone devices isn’t more complicated, per se — just different. On newer Samsung Smart TVs, press and hold the microphone button on the included remote control. And on the Family Hub, say “Hi Samsung” or “Hi Bixby” within earshot of the refrigerators’ microphone array. (To customize the wake word on any Hub, navigate to Settings > Voice > Wake Word.)

Boss Bixby around with voice commands

Bixby 2.0 — the next-gen version of Bixby that Samsung previewed at Mobile World Congress 2018 in March, and which launched alongside the Galaxy Note9 in August — has been improved in several key ways. It now has better natural language processing, faster response times, and built-in noise reduction tech that together significantly enhance its phrase and word comprehension skills.

It’s also more conversational. When you ask it about upcoming concerts around New Year’s, for example, it’ll remember the date range when looking for tickets in the future. It knows your preferences — when you request that Bixby book a restaurant, it’ll infer the size of your party and the time based on past reservations. And it can make recommendations based on your previous searches.

Bixby, like any modern voice assistant, recognizes requests to add items to your calendar, queue up tunes, place calls, and launch apps, and it can answer basic questions about sports scores, movie showtimes, business hours, and more. Here’s a sampling to try:

• “Take a selfie”
• “Take a screenshot”
• “Read out the latest messages”
• “Open an app in split-screen view”
• “Accept the call with speaker”
• “Navigate to [address]”
• “What’s on my schedule?”
• “What’s the weather forecast today?”

When it comes to reminders, there’s the self-descriptive Bixby Reminders app, which is basically a souped-up, contextual clipboard for web, email, video, and photo content. It’s where you can set notifications to arrive on a given date or when you reach a geofenced location, and where you can link to a video you plan on picking up later.

To add an item to Bixby Reminders, try saying something like “Bixby, remind me to pick up the milk, eggs, and bread when I leave work” or “Bixby, remind me to pay this month’s rent next week.” Voice, it’s worth noting, isn’t the only way to create new reminders — they can be entered directly into the Reminder app or created via Android’s native Share menu or the shortcut buttons in native Samsung apps like Phone and Messenger.

Use third-party apps

Bixby’s third-party services support isn’t as robust as, say, Alexa’s, but it’s getting better. So far, Samsung’s assistant works with dozens of apps through Bixby Labs, an early access portal for new features. Here’s a partial list:

• WhatsApp
• Facebook
• Facebook Messenger
• YouTube
• Instagram
• Twitter
• Flipboard
• CNN
• Tumblr
• Expedia
• Gmail
• Google Play Store
• Google Play Music
• Google Maps
• Uber
• Pandora
• Yelp

Now, here’s where Bixby has a leg up on the competition: It’s able to perform highly specific commands involving multiple steps. As if by magic, Bixby can automatically launch an invoked app, navigate to the appropriate screen, and complete the requested action. Here are a few examples:

• “Open Facebook and share the three most recent photos”
• “Open the gallery app in split-screen view and rotate misaligned photos”
• “Play videos on a nearby TV”
• “Open Pandora and play my ’80s station”
• “Open Uber and rate my driver 5 stars”
• “Open Instagram and post my most recent photo”
• “Open Google Play Store and install Fortnite”
• “Open Gallery and find pictures from New York City”
• ” YouTube and show my history page”

Customize the sound of Bixby Voice

Bixby’s a certified polyglot — as of publication time, it’s fluent in six languages: English, Korean, Chinese, Spanish, German, French, Italian, and Spanish. The Galaxy Note9 is a prerequisite where the last four are concerned, for now — support for Romance languages hasn’t come to the Galaxy S8, S8 Plus, S9, or S9 Plus just yet — but Samsung says it plans to expand device support to more languages “over time gradually.”

No matter which device you’ve got, switching Bixby’s native tongue isn’t terribly complicated. Launch Bixby Home, and then tap the More Options button. Navigate to Settings > Language and Speaking Style, touch Languages, and select your dialect preference.

That menu’s also where you can select a different speaking style for Bixby if you so choose. Within the Language and Speaking Style menu, tap one of the three options: Stephanie, John, or Julia.

See a history of commands

Ever wonder which requests you made of Bixby yesterday? The day before? A week ago? Good news: Samsung’s assistant has a nifty feature — Bixby History — that saves up to seven days’ worth of commands (and replies) for posterity. It also lets you delete commands, or add a Quick Command, a phrase that kicks off a multistep process. (For example, “Create my emoji” could launch the camera app and pull up Samsung’s AR Emoji tool in one fell swoop.)

To access Bixby History, tap the physical Bixby button. Next, head to More Options > My Bixby > History, and voila — you’ll see a list of utterances in chronological order, along with quick links (Suggest another phrase) to correct any misinterpretations.

Zapping commands from the list is easy — tap the Delete button in the upper-right-hand corner of the screen and the radio icons next to items you’d like erased from Bixby History. To add Quick Commands, meanwhile, touch the Add quick command button under the utterance you’d like to associate with a shorter word or phrase. It’ll prompt you to say the new Quick Command, after which you’ll be able to save it by touching the Save button, or re-record it.

Control smart home devices

Bixby Voice plays nicely, unsurprisingly, with Samsung’s SmartThings platform of connected wireless lightbulbs, deadbolts, ovens, thermostats, dishwashers, dryers, and other smart home devices.

On the Galaxy S8, S8 Plus, S9, S9 Plus, and Note9, you’ll need the SmartThings app to take advantage. It should be preinstalled, but if for some reason you can’t find it, search for it in the Google Play Store or the Samsung Galaxy Apps marketplace and install it.

When you launch the SmartThings app for the first time, pop-up windows will alert you to connect all detected SmartThings devices. Do that, and you’ll be ready to start orchestrating SmartThings devices with voice.

With Bixby Voice and SmartThings, you can add modes, activate appliance functions, change the color of lights, check a sensor’s battery level, turn down the temperature, and more, providing you’ve got the requisite hardware. Give these commands a go:

• “Show my devices at home”
• “Search for new devices”
• “Set the temperature to 65 degrees”
• “Turn on the TV”
• “Turn on Power Cool for the fridge”
• “Run the Good Morning scene in the bedroom”
• “Add a mode to home”

3 things to try with Bixby Vision

Bixby Vision, much like Google’s eponymous Google Lens or Amazon’s Flow, taps computer vision to recognize and classify objects in photos.

Thanks to partnerships with Vivino, Amazon, Nordstrom, Sephora, Cover Girl, Elle, Laneige, FourSquare, and Pinterest, Bixby Vision can scan QR codes and show relevant product listings, recommend wine, and display the calorie counts of food. It can translate text and let you virtually “try on” makeup products — and, thanks to Adobe’s integrated document scanning technology, extract text from receipts and business cards.

Bixby Vision is only supported on the Galaxy S8, S8 Plus, S9, S9 Plus, and Note9. There are a couple of ways to launch it, but the easiest is by tapping the Bixby button in the camera app or the Bixby Vision icon on the left side, or by touching the Bixby Vision shortcut in the Gallery app.

Translate text

If you’re traveling abroad, what better way to suss out the meaning of foreign signage than with your phone?

Bixby Vision’s augmented reality translation works more or less like comparable tools in Google Translate and Microsoft Translate. Simply fire up Bixby Vision, select the target language, and line up the text you want transcribed in the on-screen box. You’ll get an overlayed translation in real time, if all goes well.

Want to convert that text to an editable format? No problem. Within Bixby Visions’s Text Mode, look for the circular logo underneath containing the letter “T.” Tap it, and Bixby will do its darndest to extract the print or handwriting.

Try on makeup

Can’t be bothered to trade your comfy PJs for jeans, but itching to find a new favorite lipstick, eyeliner, foundation, or mascara? Not to worry. Bixby Vision — with content providers Cover Girl, Elle, Laneige, and Sephora — has you covered, thanks to tech from L’Oréal company ModiFace.

From within Bixby Vision’s SnapChat lens-like Makeup Mode, you’ll see a number of brands to choose from. Select one, and tap a product from the carousel to digitally apply a product to any face within view of your phone’s front-facing camera. You can “try on” products individually or choose from completed makeup looks that include lipstick, eyeliner, blush, and more.

If the mix isn’t to your liking, starting fresh isn’t as arduous as rubbing off the real stuff — simply tap the circular arrow icon. And when you find something you like, you’ll be able to purchase it through Bixby Vision. Snapping a selfie saves it to your gallery and surfaces info about the products you “tried on,” including brand names, descriptions, price, and a link to the online listing.

Compare wines and shop for products

If foundation isn’t your thing, how about something potentially less fraught — like libations? Thanks to Samsung’s ongoing collaboration with wine database Vivino, a quick label scan is all it takes to yield reviews, grower and grape info, and more.

And if wine doesn’t float your boat, how about Bixby Vision’s product search feature — Bixby Shopping — that incorporates retailers like Amazon, Lowes, Sam’s Club, Walmart, and Nordstrom. When Bixby Vision spots clothing, shoes, movies, home decor, books, or other things it recognizes, tapping the shopping cart icon surfaces matching results.

Count calories

Is that salad you ate for lunch really healthy, or are there hundreds of empty calories lurking in the ranch dressing? Bixby Vision promises to get to the bottom of things — in theory, at least.

Bixby Vision’s calorie-counting feature can identify foodstuff like steak, pizza, salmon, and sushi and match it with a generic listing in Samsung’s nutritional database. The company makes no claims that it’ll chip away at your waistline — Bixby Vision can’t identify multiple items on a plate, and it doesn’t always guess entrees correctly — but the data’s helpfully ingested by S Health, Samsung’s wellness and activity-tracking app, where it’s shown alongside your estimated daily calorie burn.

Bonus round: Bixby Home

We’d be remiss if we didn’t briefly mention Bixby Home, a “social stream for your device.” It’s a dashboard of reminders and social media updates collated in cards that can be dismissed, pinned, or permanently hidden.

By default, Bixby Home shows upcoming appointments, photos you’ve snapped this week, tomorrow’s weather, the number of steps you’ve taken, breaking news events, Facebook status updates, YouTube videos, new LinkedIn posts, and more. According to Samsung, it looks for patterns in your app usage by time and location to predict what you’re looking for — so when you head to the gym, for example, you might see your playlist and Samsung Health.

Bixby Home doesn’t match alternatives like Google Now when it comes to app support, but major players like Twitter, Spotify, and Flipboard are present and accounted for. Each displays swipeable “cards” containing highlights like trends, recommendations, and recent activity.

Home can be customized, to a degree. Head to Bixby Home and tap the Menu Button in the upper-right-hand corner, and then touch Settings. Tap Cards, and then the toggles corresponding with the card you’d like to remove from Bixby Home.

To pin a card so that it’s always visible, tap the Menu Button on the card you want to pin and touch Pin to Top. (To unpin it, select the Menu Button and tap Unpin.) To change Bixby Home’s background, tap the Bixby Home Menu Button, touch Settings, and select Background color. Select the color you want to use, and tap Apply.

If you’d rather not bother with Bixby Home, you can disable it altogether by pressing and holding anywhere on your phone’s home screen, swiping to the right to reach the left home panel, and switching the toggle next to Bixby Home.