Fractile raises $220M to build next generation of inference hardware

UK-based AI chip startup Fractile has raised USD 220 million to accelerate the path to getting our first chips and systems into customers’ hands, in a financing round led by Accel, Factorial Funds, and Founders Fund, with participation from Conviction, Gigascale, 01A, Felicis, Buckley Ventures and 8VC, alongside its existing backers. 

Fractile was founded in 2022 on the bet that, eventually, the world’s most capable AI systems would be limited in their impact by the amount of time they take to produce useful outputs, the company said. 

“We bet everything on the logical conclusion: that the only way to truly unlock this latent value, to make speed viable at scale, was to radically re-invent the hardware that we run our frontier AI models on,” Fractile said. “Ever since, we have been building chips and systems that tackle this problem.” 

Today’s LLMs are already producing up to 100 million tokens in pursuit of tackling these hard problems. At the ~40 tokens per second or so at which these models tend to run on existing chips, a single output of this length takes a month to complete. 

“The technical and economic limits on inference speed, above all from memory bandwidth that has failed to scale on current architectures, are what is constraining progress. To compress that month into a day, we will need to generate output at ~1,200 tokens per second, while handling the complexity and capacity challenges of operating large models at very long contexts,” the company said. “This is exactly the problem Fractile has been building from the ground up to tackle.”