The Future of Food: How Smart Ovens Will Transform Cooking Forever

Little did Jeff Donnell know that he'd be taking hundreds of photos of raw chicken. In order to teach Whirlpool's new smart oven what uncooked poultry looked like, the product-development manager for WLabs — Whirlpool's team tasked with creating innovative devices — had to snap picture after picture of fowl, both frozen and fresh.

The smart oven being developed by Whirlpool ($800) and coming in August is one of a new generation of kitchen appliances that will not only identify what you're trying to make, but also show you the best way to cook it. Using a combination of machine vision and cooking algorithms, these devices can help harried parents get dinner on the table faster and get others out of the rut of making the same five meals.

And because these smart ovens are connected to the cloud, it's easier for the company to diagnose a problem when your dinner goes up in smoke. Here's what went in to creating these next-gen ovens and how soon you might see one in your own kitchen.

What the heck is a smart oven, anyway?

From the outside, a smart oven looks like an oversize toaster oven; you're definitely going to need to free up some counter space. The idea behind all of these smart ovens is to simplify the cooking process.

Like with most "smart" appliances, what separates smart ovens from more-conventional countertop ovens is their connectedness; hook them up to your Wi-Fi network, and you can control them using a smartphone app. In addition to alerting you when your food is ready, such an app also comes packed with recipes that can automatically tell the oven the correct temperature and time for a meal.

These ovens, which use heating elements at the top and the bottom of their chambers, can perform a number of tasks, such as baking, roasting, air frying, dehydrating, proofing and reheating food.

But the killer feature in a next-generation device like this is the camera inside the oven. At their most basic, these cameras let you peek inside the oven using your phone to see if that steak is browned, without having to open the oven door and let heat escape. 

The cameras serve a much more important function, though; they connect to a database in the cloud and can not only determine what you've put in the oven, but also tell you how long to cook it and when it's done. 

Whirlpool's $800 smart oven is not the first of its kind. There are already two other smart ovens on the market: Brava ($1,095), which went on sale in December 2018, and the June ($600), which launched in 2016 and released its second version last summer. Both of those also have interior cameras; June uses its camera in a similar manner as Whirlpool's, while the Brava oven uses its camera to measure the "doneness" of your food.  

"If you think about it, computer vision and food ID is the best search engine in the world," said Matt Van Horn, the CEO and co-founder of June. 

"It's faster than voice, faster than a list. It's faster than typing," he said. "You take a nice piece of steak or salmon and put it in the June, and the touch screen says, 'I see you put in a steak — how would you like it cooked? You did medium rare last time. Do you want to do that again?' And so we're able to learn from all the June ovens in the world to create this experience."

Cooking with algorithms

While there are a lot of photos online of what a finished meal looks like, no one takes pictures of what food looks like before it's cooked. That's why Donnell and his team had to photograph raw chicken as well as all sorts of other food, cooked and otherwise. He explained that consumers who are novices or simply pressed for time may lack the confidence to decide how long to cook a frozen chicken.

"That's where we said, 'Let the oven take care of that for you.' We'll develop the algorithm that understands, yes, this is a frozen chicken breast and, yes, I know how to cook it."

But identifying what you're about to cook is merely the first step with a smart oven. You've got to make sure the food comes out properly every time, which is where all the sensors in the ovens come into play. 

"It's very different from cook to cook," said John Pleasants, CEO and co-founder of Brava, whose oven has 10 sensors: a camera, as well as a particle sensor, a humidity sensor, two temperature sensors and a probe with five thermometers. 

"We knew we had to have feedback loops, because a steak from Safeway is going to be different than a steak from Trader Joe's," said Pleasants. "We needed to know what was going on during a cook and course correct." 

Because they're using algorithms, the ovens can be as fussy as a sous-chef whipping up a hollandaise. "The difference in developing an algorithm is that we're able to get more fine-tuned about the beginning, middle and end of the process," Donnell said. 

So, for instance, an algorithm for making a pizza could specify convection baking at 375 degrees Fahrenheit for 5 minutes, a standard bake at 400 F for 5 minutes, back to convection bake and then broil for 30 seconds. "The user is not going to go through those steps manually," he said.

Instant feedback

Because these ovens are cloud-connected, that also means companies can get instant feedback from home cooks when their meals don't come out to their liking. If something's amiss, users can send feedback from the oven's app, and the company can look at both sensor data and visual data from the ovens to analyze what went wrong. They can then push an update over the air to fix a faulty recipe. 

"If you give us a thumbs-down, we ask two really simple questions: How was the inside, and how was the outside?" Donnell said. "And what that enables us to do is triangulate between different points of data to go and improve the performance." 

Van Horn discovered this early with "Bacongate." At the time, the June oven had just one cooking program for bacon, but customers complained that the oven was overcooking and undercooking bacon. After poring over all the video logs and sensor data, June's engineering and culinary teams realized one variable they hadn't considered: the number of slices of bacon. 

"At that time, we had a one-size-fits-all bacon program," Van Horn said. "And unless it was [done] in the exact way we designed the cook program, a lot of people were getting bad results." 

Now, the June oven takes into account four different criteria — number of slices, thin or thick cut, chewy or crispy, and using tinfoil or not — before deciding what cooking algorithm to use. Now, with more than 64 bacon programs on the oven, he said, "we rarely get complaints about bacon anymore."

Brava's smart oven employs a similar feedback loop.

"If you had an oven and were to call in and say, 'I wasn't exactly thrilled with the way my chicken thighs came out,' we have a video of the entire cook," Pleasants said. "We see how you put it in, all the data of how the temperatures were rising, and we can see if the chicken was on the wrong shelf."

Whirlpool's Donnell said that the feedback can be used to tailor the ovens toward regional preferences, too. 

"You've got pizza ratings in New Jersey [that] are coming back negative consistently, but the pizza ratings in Chicago are coming back positive," he said. "What we're able to do is split that into two different algorithms and redefine what that New Jersey, East Coast algorithm looks like and push an update to that geography specifically." 

June also used data from its oven to figure out how to reduce the price of the second-generation version by more than half. 

"We had four load cells, weight scales on the original June oven, and that was an expensive feature," said Van Horn. "We were able to look at how often people were using the weight scale app, and it was not a commonly used app, so we were able to remove that, and I don't think I've personally received one complaint." 

While That wasn't the only cost-cutting measure the company took, it helped decrease the June's price to $600 from $1,500.

What's next?

Not surprisingly, the founders of June and Brava are bullish that their products will be the ovens of the future. "It's going to cannibalize the use of your big oven and your stovetop to cook better, healthier, tastier food," Pleasants said.

Whirlpool is a bit more measured in its approach, offering only 2,000 smart ovens in its initial release. However, Donnell said that what's successful in WLabs will get incorporated into other parts of the company. 

"What are we going to learn from the early adopters using the product to really change the kitchen landscape in the next two, three, five, 10 years?" he asked. "AI in the kitchen is not going to be an unfamiliar space in the near future. So, it was up to us to figure out what play are we going to make in that space that we could do quickly, what we could learn from a lot, get a lot of information [from] fast and that we can extend into the rest of our product ecosystem." 

The challenge, as with all smart home products, comes not only in conveying what it does, but also in illustrating the device's benefits to consumers. And an $800 smart oven is a lot more expensive than a smart light bulb. 

"Normal people will say, 'What the hell do I need a camera in the oven for?'" said Ryan Herd, a National Kitchen & Bath Association "insider," who acts as a sort of ambassador for the group. "They just want to know that their cookies and brownies are going to be perfect each and every time." 

In the future, Herd said, he imagines that connected kitchen appliances will not only be able to set the temperature automatically, but also come with instructional videos from famous chefs who will guide you through the cooking process, Peloton-style. 

However, he cautioned that the current crop of smart ovens are proof-of-concept at this point. "When that is baked in to a real oven that's not taking up our tabletop," he said, "then we're really going to benefit." 

Blake Kozak, the principal analyst for smart home and security technology for IHS Markit, estimated that full-scale adoption of smart kitchen appliances is inevitable, but still a good 10 years away. And there could be other complications if you want all your appliances to talk to each other, he said. Given that you can't get Alexa and Google devices to always play nice, how can you expect the same out of the major appliance makers?

"Microwave ovens and other small appliances could be a way to introduce connected technology to the oven," Kozak said. "But then if the consumer really enjoys the experience, they could be left in a difficult position if they want to connect that appliance to a GE or LG or Samsung refrigerator and other appliances."

But as connected appliances take over, the technology has to augment and not intrude on what's a very emotional aspect of home life, Pleasant said. "It's almost like we're building an extra pair of hands in the kitchen," he said. "When you go to the kitchen, you're not alone."