Alongside the announcements for the 4th generation Nest Learning Thermostat and the Google TV Streamer, the search giant also revealed some improvements coming to the Google Home app that will leverage AI to make using some of the best home security cameras, video doorbells, and other smart home devices a lot easier.
Coming this fall — Google wouldn't specify exactly when — Gemini will be used to analyze videos connected to the Google Home app, and generate detailed summaries of each clip. So, for example, rather than a clip simply identifying an animal in your yard, it will be able to create a summary saying "dog peeing on your flowers."
As a result, Google Home users should also be able to search through their video clips more easily using natural-language queries, such as "when did my child leave their bike in the front yard?"
That way, you should find it easier to sift through the hundreds of hours of saved video to track down the exact clip you're looking for.
Of course, using an AI to analyze your videos does raise a number of privacy concerns; in its announcement post, Google reiterated its commitment to privacy and security.
(Update Aug. 7): In response to some questions by Tom's Guide, a Google spokesperson said that the company would not use videos or data from consumers to train Gemini models "except where users have explicitly consented to their videos being used."
We also asked if you would be able to opt out of this feature on a camera-by-camera basis, but the company said that would not be possible initially. "We continually look for feedback from users and will iterate on the experience based on what’s most impactful for users," the company said.
Smarter smart home automations
In addition, Google said it's extending Gemini to other parts of the Google Home app, to make it easier to create home automations, even ones that are more complex. So, instead of using the script editor to create a routine, you could type in a phrase such as "turn on the lights and set the thermostat to 72 degrees when I arrive home," and the app will set everything up for you.
Gemini will also be able to suggest routines based on what you can do with your smart home devices, which could help you to get more out of what you already own. One example cited by Google is “Help the kids remember to put their bikes away when they get home from school,” which would broadcast a reminder over your Google smart speaker when your child arrives home.
A new Assistant voice
Similar to what Amazon is working on with Remarkable Alexa, Google is also trying to make its assistant more conversational in everyday speech, so that when you ask it a question, it will respond in a less robotic manner.
And, you'll also be able to ask Google more complex questions, such as “Play that song that Ice Spice played with Taylor Swift during the Eras tour."
I heard a brief demo, in which Google Assistant was asked about Pluto, and the new and improved Assistant had much more intonation and feeling in its voice than the current model.
Coming soon to a smart speaker near you
Google said that all of these new smart features would roll out to Nest Aware subscribers as a public preview later this year, but did not specify when they would be made available to Google Home users in general.
I am interested to see how well these new features work — there's nothing more tedious than going through hours and hours of security camera footage to try and find one specific event — but I am nervous as to how my videos will be used. Given that a number of companies used YouTube to train their AIs without peoples' knowledge, I'd be wary of giving permission for more personal videos to be used without some strict guardrails in place.
And, as we're still waiting for Remarkable Alexa to appear, I'm curious to see how Amazon's revamped assistant will compare to this new and improved Google Assistant and which will be the best for your smart home.
More from Tom's Guide
