We are currently in an age of AI, at least when it comes to our phones. With the recent revelation of Apple Intelligence, there is now no smartphone that does not come with at least some form of generative AI. However, I have noticed something troubling when it comes to Google’s own Gemini.
Recently, I started testing three different generative AIs to create D&D backstories, and that led to me playing around with them to see what they could do. One of the tasks I set for the AI was to create a 2K Warhammer 40K list for an Ork army in 10th edition. In theory, Gemini would go through the web to find the point values and tactics for the faction in question, and then present the best option to me.
However, that isn't what has been happening and the information and lists I was given were completely out of date. This is a big issue for me and many other users. However, before we get into that we need to first understand why Gemini is giving out such old information.
Why is Gemini giving out-of-date info?
The first thing to know is that, when it comes to Gemini, there are actually three variations. The one that we see on mobile devices is Gemini Nano, and this AI is not connected to the internet in the same way as the other two and, as such, its training set is slightly different.
Generative AI is often trained, meaning that the developers introduce data to the AI model and slowly shorten the gap between the AI’s actual output and the expected output. This training, which can include updated information, is introduced to Gemini Nano in a “Training round” every three months. This information is crucial as it allows Nano to respond to questions with up-to-date information and, if the training round is lacking, then Nano can only respond with the information it has available.
Another issue is that Google Gemini supposedly has considerably fewer parameters than other models of its ilk. It has been stated that the Gemini has about 1.8 billion parameters, about half of Apple Intelligence. And let's not forget that more models are being released every day. These parameters determine how the model behaves, and the more that it has the more the model can process data.
The issue that has arisen is that the information being provided is out of date, meaning that the updates for specific subjects have been missed. Now, if this issue only occurred with Nano it would be somewhat understandable, but the problem is that the same misinformation happened on the web browser version that is connected online.
Why is Gemini Nano giving out old info an issue?
This is an issue for several reasons, but the main one has to do with the purpose of an AI like Gemini. In concept, Google wants a chatbot that will be able to answer any question you have and improve how you search the web. Now, if I want to know what the weather will be like then even Nano appears to work fine. However, I don't search the web for info like that, I search the web to look up my interests and hobbies and Google Chrome is getting better every day.
Let's take Warhammer as an example again: I want to build a new army and gauge how much it will cost on average. Now, If I were to do that solo I would need to build the list on one app, search through the prices on another, and possibly have a website with some tactical help to drive my choices. That's three different sources I need to manage. In theory, the AI should be able to help me by gathering all that information in one place and then presenting it to me in an easy summary. If Gemini can't do that, then why would I download it?
Now, let's say I take the out-of-date information and budget and plan around it, only to find that it's effectively redundant as far as my hobby goes. This becomes a major problem and would lead me to abandon Gemini, which Google has been pushing on nearly all devices. In the end, this means that Gemini isn't a useful tool, at best it has become a gimmick.
So, what can Google do about this?
As I previously stated, Google wants to include Gemini on all of its devices as well as have users download the feature. As such, it needs to solve this issue otherwise there is no reason for users to choose it over the competition. This is especially problematic when you note that Apple is working to integrate ChatGPT into its Apple Intelligence, with plans to add more.
The truth is that Google Gemini on phones is still relatively new and still needs a lot of work to be perfect. Sadly, it all feels too little too premature for me, and I think I'll just keep searching Google myself for the time being and trust my own fact-checking. Although hopefully we might see a change soon, AI is ever-changing after all.
More from Tom's Guide
- AI is better at fixing itself than with human help — meet OpenAI's new bug hunter
- OpenAI shares a new GPT-4o advanced voice demo — it can teach you a language
- GPT-4o advanced voice 'accidentally' leaked out to some users — here's what happened
