AI giants like Google, Meta and OpenAI may grab all the headlines, but behind the scenes there’s an equally turbulent and rapidly expanding ecosystem taking shape.
The market for small and specialized AI models that can run on home computers is reaching fever pitch, driven by a seemingly limitless demand for personal and business applications.
Sparked two years ago by the launch of Meta’s open source Llama model — and ignited into a frenzy by the release of DeepSeek R1 this year — this homebrew AI sector looks to be on an unstoppable trajectory.
It’s not only that these local models are cheaper and more private, they're also proving to be easy to customize for just about any kind of purpose.
But are they really that useful, or is it all just wishful thinking? I thought it might be worth looking at three of the main pretenders to see what they offer.
DeepSeek
It’s probably fair to say that no model has done more to accelerate the local AI sector than this shock Chinese product. Free, open source and extremely powerful, it’s a perfect tool for anybody to want to experiment with new AI applications.
There are two major reasons for its explosive success. First it can run on extremely modest hardware, especially in its smaller versions. Second, it can easily be used to train other models to produce powerful AI model hybrids in a process known as AI distillation.
My current favorite is DeepSeek R1 Distill Llama 8B, which at 5.3 GB in size is small enough to run on my desktop PC, but offers a good solid range of performance to cope with most day-to-day tasks.
My current favorite is DeepSeek R1 Distill Llama 8B, which at 5.3 GB in size is small enough to run on my desktop PC, but offers a good solid range of performance to cope with most day-to-day tasks.
This ranges from basic chat search, e.g. how can I remove stains from a cotton T-shirt, to handling tax queries or other personal issues.
Because it runs locally on my computer and doesn’t need an internet connection, I can be assured of my privacy, which is nice.
Qwen
Another good option is the Qwen range of models.
I currently have three versions of Qwen 2.5 on my PC, specifically the 7B, 14B and 32B models. Only the smallest really runs at an acceptable speed on my machine, but occasionally I use the other more powerful versions if I’m feeling patient enough to wait around for the response.
There’s also a neat coding version, which offers free code generation for creating small simple apps and utilities.
Llama
The pioneering Llama has proved to be a robust, reliable and very flexible model for various uses.
The one area where it's still especially strong, is vision. So I run Llama 3.2-vision to scan documents and decipher images. This sounds silly, but there are hundreds if not thousands of applications using this model, from scanning car VIN plates to radiology.
I also have a custom tuned version of Llama 3 which I love using for general knowledge. It seems to consistently deliver more detailed and accurate responses per question.
Points to note
There are a few things to note about using local models. The first is newer is almost always better. AI development is progressing at such high speed that even six months can mean a huge difference in quality and performance.
It's also important to understand that the use of local models means you’re inevitably going to suffer from a smaller context window — that is the ability to handle large chunks of text in one go, unless your computer has a significant amount of memory and a powerful graphics card.
This can restrict their usefulness for more complex tasks, but is also slowly changing as the tech matures.
Bottom line
There’s now a huge number of open source models on the market, so there should be something for everyone.
A great place to start is by doing a search on the open source model catalog at Hugging Face. Most models can be installed and run from Ollama or the LMStudio app.
More from Tom's Guide
- Finally! RTX 5090 leak points to 'stupidly high' stock coming next month
- Help improve Tom's Guide — your chance to win a $250 Amazon gift card
- Buying cheap tech could take a hit in 2025 — Trump’s tariffs, explained
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
