New artificial intelligence models, tools and platforms seem to launch every day but few have gained the hype I’ve seen surrounding Groq, and that hype seems well deserved.
Founded by a poolside in 2016, Groq builds custom chips specifically designed to run large language models such as the one underlying OpenAI’s ChatGPT or Google’s Gemini.
It currently has access to several open source models including Meta’s Llama 2 and French AI unicorn Mistral’s family of models including the near-GPT-4 level Mixtral.
The company says it can respond to queries 75 times faster than the average human can type and significantly faster than anyone can read — upwards of 100 tokens per second. This is hard to compare but is faster than running on a GPU or CPU.
How to use Groq
It might have a similar name to Elon Musk’s Grok chatbot that resides in the background of X but Groq has no AI model of its own, instead providing rapid access to leading models.
Speed isn’t always essential for AI tools to be usable but if you want to get the most natural conversation going with the bot then you want a rapid response.
1. Visit the Groq website
For now the best way to use Groq is to visit the website of the AI platform, sign up and start chatting. To get started go to groq.com.
2. Sign-in with Google
Currently the only option to sign-in is with Google, although you can use the platform for a while without having to sign-in.
To sign-in click on the Welcome button on the top right corner then click Login. This will take you to the Google login page where you can just sign-up with your existing Google account.
3. Select a model
When using ChatGPT, assuming you pay for Plus, you have the option of switching between GPT-3.5 and GPT-4.
Groq works in a similar way, allowing you to switch between different models and different versions of models. To change the model just click on the model name on the top-left of the window.
The final 2-3 characters determine the size of the context window. For example, 32k means the AI can hold up to 32,000 tokens, or about 24,000 words before it forgets the start of the message.
4. Start asking your question
Groq works just like Gemini, Copilot, Claude or ChatGPT. You have a prompt box where you can type the message of your choice and then the response will appear.
To get started just type your prompt and hit Enter or click the red arrow.
5. Read the AI response
The response from the AI, as well as how fast it generated that response will appear under the prompt box. You can read it, ask a follow up question or modify the response.
6. Modify the response
To modify the response click the Modify button. This is a clever feature that I’ve not seen to this extent in a chatbot before. It is available in Docs or Word but this is a new one for a chatbot tool. You can click Modify and select from different layout options. This effectively refines the original prompt, such as putting it in a table or as a list.
7. Set a system prompt
A system level prompt is similar to custom instructions in ChatGPT, it is a set of core instructions the AI has to adhere to in any message response. This could be used, for example, to tell it to act like a strict lecturer or a gentle grandmother. You can also give it core information such as your name statistic you need it to hold in its memory longer than a single chat.
To access it click the cog icon in the top right and select Sys Prompt.
8. Copy the content of a message
You can send multiple messages and copy the content of each individual response by clicking on the copy icon next to the Modify button. You can also copy your initial message.
For more tutorials on using AI tools check out our guide to creating images using Google search, a guide to making video with Runway and how to ask YouTube questions about videos.
