Forget ChatGPT — Groq is the new AI platform to beat with blistering computation speed

News
By
published

AI has a need for speed that Groq wants to deliver

An AI prompt box on a futuristic background
(Image credit: Shutterstock)

Groq, a company that created custom hardware designed for running AI language models, is on a mission to deliver faster AI — 75 times faster than the average human can type to be precise.

Speed is very important with it comes to using AI. When you’re having a conversation with an AI chatbot you want that information to happen in real time. If you’re asking it to compose an email you want the results in seconds so that you can send it off and move on to the next task.

Groq (not to be confused with Elon Musk’s Grok chatbot — and no they aren’t too happy with the similar names) specializes in developing high-performance processors and software solutions for AI, machine learning (ML), and high-performance computing applications.

So while the Mountain View-based company (currently) doesn’t train its own AI language models it can make ones developed by others work really fast.

How does it achieve this?

Groq uses different hardware than its competition. And the hardware they use has been designed for the software they run, rather than the other way around.

They built chips that they’re calling language processing units (LPUs) that are designed for working with large language modes (LLMs). Other AI tools usually use graphics processing units (GPUs) which, as their name implies, are optimized for parallel graphics processing.

Even if they’re running chatbots, AI companies have been using GPUs because they can perform technical calculations quickly and are generally quite efficient. Building on the example of chatbots, LLMs such as GPT-3 (one of the models that ChatGPT uses) work by analyzing prompts and creating text for you based on a series of predictions about which subsequent word should follow the one that comes before it.

Since Groq’s LMUs are specifically designed to deal with sequences of data (think DNA, music, code, natural language) they perform much better than GPUs. The company claims its users are already using its engine and API to run LLMs at speeds up to 10 times faster than GPU-based alternatives.

Try it out

You can try it out for yourself for free and without installing any software here using regular text prompts. 

Groq currently runs Llama 2 (created by Meta), Mixtral-8x7b, and Mistral 7B.

On X, Tom Ellis, who works at Groq, said custom models are in the works but that they’re concentrating on building out their open source model offerings for now.

More from Tom's Guide

Christoph Schwaiger
Christoph Schwaiger

Christoph Schwaiger is a journalist who mainly covers technology, science, and current affairs. His stories have appeared in Tom's Guide, New Scientist, Live Science, and other established publications. Always up for joining a good discussion, Christoph enjoys speaking at events or to other journalists and has appeared on LBC and Times Radio among other outlets. He believes in giving back to the community and has served on different consultative councils. He was also a National President for Junior Chamber International (JCI), a global organization founded in the USA. You can follow him on Twitter @cschwaigermt.

2 Comments Comment from the forums
  • Scott_21
    How can they get away with using the same name basically?
    Reply
  • RyanMorrison
    Scott_21 said:
    How can they get away with using the same name basically?
    The word Grok dates back to the early 60s, first coined in Heinlein's Stranger in a Strange Land. It had multiple meanings, one of which was to understand — hence the fascinating with the word among the emerging tech scene over the next few decades.

    The AI chip/platform Groq actually predates Elon Musk giving his chatbot the name Grok by 7 years as it was founded in 2016 and Grok launched last year.
    Reply