Big name AI may get all the headlines, but behind the scenes there’s another revolution going on. AI chatbots are slowly but surely moving onto the smartphone – not connected to the cloud, but running locally and privately on the device itself. I recently spent some time testing out three new free AI chat apps for Android, to find out whether they’re worth installing.
The driver behind this move to mobile devices is the rise and rise of extremely capable open source AI models like Llama 3.2, which can now approach the performance of large models but in a much smaller footprint.
It’s still early days for these apps, but from what I’m seeing this is something that’s going to really change the way we access information in the future. My tests revealed three surprisingly good chatbots. Something that's all the more remarkable with the meager resources of my Android phone. I was genuinely shocked by how satisfying they are to use.
1. Local AI
This chatbot has an interesting take on adding AI to a phone. Instead of offering complicated setups and installation, the app gives a limited choice of AI settings, which keeps things super easy to understand for newcomers. Installation is quick and easy, and the only decision the user has to make is whether to use the Fast, Balanced, or High Accuracy settings, which represent the AI in use.
The trade-off is clearly speed versus sophistication, but the one thing to say is the idea seems to work very well. For instance the Fast setting is extremely quick. It’s not as fast as ChatGPT, but when you consider this is running on a sliver of a smartphone processor with minimal memory, it’s nothing short of a miracle how quickly the responses come up on screen.
The UI is easy to use, just a chat box and response screen, and one great feature is the fact you can press a microphone icon and talk to the model instead of having to laboriously type out your request. The only downside is the long press copy paste function to grab the AI response text seems to be broken. Hopefully that will be fixed soon by the developers.
Download Local AI on the Play Store here.
2. Private AI
Private AI takes a more conventional approach by offering configuration options so the user can tailor the app to their needs. This includes downloading a wide choice of open source models onto the phone, like Mistral, Gemma, Phi, TinyLlama and suchlike.
However note that if you have a standard spec smartphone, you’re not going to want to download any model over 1 to 3 GB in size. Beyond that, things run very slowly indeed.
The model size vs speed equation is a constant constraint of these apps. The fact is current smartphones, apart from very high end models, are not built for processing AI. The result is most of the responses will be quite slow even using tiny models such as Gemma 2. I tested it on a Samsung Galaxy S20, and even at a modest 1.9 gigabytes in size it was still quite slow.
The app also lacks basic amenities such as deleting past chats from the history, and is relatively primitive compared to the other apps in this test.
Download Private AI on the Play Store here.
3. PocketPal
One of the big surprises was how good this free Android app is. What makes it special is the fact it seems to optimize the open source models better than the other chatbots. I was able to load Llama 3.2 3B into the app and get excellent performance out of it, better than the other products. This is a two gigabyte sized model, and is probably one of the best in class for its size anywhere, whether on desktop PC or mobile. So the fast speed is a surprising bonus.
Like Private AI, PocketPal also offers a number of user customization options to tune it to your particular needs. This includes different cache settings, and also using flash attention for faster processing.
One feature I particularly liked is the ability to add a different system prompt to each model individually. I thought I’d have fun and set my model up to talk like a London Cockney, which was hilarious for a while.
All in all I found PocketPal to be the most mature app on the Android platform at the moment, delivering a nicely grown-up AI chatbot experience on the phone.
More of these apps are arriving on both Android and Apple handsets every month, so this is an area that clearly has a solid future as the tech matures.
Download PocketPal on the Play Store here.
More from Tom's Guide
