Groq aims to democratize inference computing with a massive funding boost, plans for hardware expansion
The startup, known for its GroqCloud platform — a cloud-based service where developers can test and develop AI models at high speeds — plans to use the funds to significantly expand its Language Processing Unit (LPU) infrastructure.
Groq has raised $640 million in a series D funding round to accelerate the deployment of its AI inference computing technology.
BlackRock Private Equity Partners led the round, which included participation from Cisco Investments, Samsung Catalyst Fund and Neuberger Berman.
The series D round brings Groq’s valuation to $2.8 billion.
GlobalFoundries is manufacturing the LPUs which will be completed by the end of Q1 2025.
Groq will also look to scale capacity for its tokens-as-a-service (TaaS) offering and add new models and features to GroqCloud.
The startup plans to add more than 100,000 additional of its custom LPUs to power GroqCloud in what it claims will be the largest AI inference compute deployment for a non-hyperscaler.
It also plans to “significantly expand” its team, including silicon engineers, supply chain operations and sales managers.
Groq was founded in 2016 by Ross, who previously led Google’s custom hardware efforts, helping design its Tensor Processing Units (TPUs).
The startup developed its own hardware optimized for running AI and software workloads.
“You can’t power AI without inference compute,” said Jonathan Ross, Groq’s founder and CEO.“We intend to make the resources available so that anyone can create cutting-edge AI products, not just the largest tech companies. Training AI models is solved, now it’s time to deploy these models so the world can use them.”
Groq claims its LPUs can run sizable models like Meta’s Llama 2 70B at over 300 tokens per second per user.
The startup provides support for a wide range of models, including the new Llama 3.1 from Meta and smaller systems like Gemma from Google, all while maintaining high speeds.
More than 360,000 developers use GroqCloud to power their training efforts.
“Samsung Catalyst Fund is excited to support Groq,” said Marco Chisari, head of Samsung’s Semiconductor Innovation Center and EVP of Samsung Electronics.
“We are highly impressed by Groq’s disruptive compute architecture and their software-first approach. Groq’s record-breaking speed and near-instant generative AI inference performance leads the market.”
More than 360,000 developers use GroqCloud to power their training efforts.
To ensure its platform keeps its fast pace for next-generation AI training, Groq has secured a new technical advisor company, Yann LeCun, Meta’s chief AI scientist and a Turing Award winner.
“Samsung Catalyst Fund is excited to support Groq,” said Marco Chisari, head of Samsung’s Semiconductor Innovation Center and EVP of Samsung Electronics.
“We are highly impressed by Groq’s disruptive compute architecture and their software-first approach. Groq’s record-breaking speed and near-instant generative AI inference performance leads the market.”
LeCun will still retain his role at Meta. His involvement with Groq is notable given Meta is working on developing its own custom hardware, MTIAs. However, these chips largely power deep learning recommendation models and not generative workloads. Despite this, Meta hopes to eventually expand the chips to handle additional capabilities.
Stuart Pann, former senior executive from HP and Intel, is taking on a permanent position as Groq’s chief operating officer.
Read More-Quantum Computing is Developing Faster Than Expected — QuEra Survey
