ZML, a French AI startup backed by Turing Award winner Yann LeCun, has launched LLMD, a free software tool designed to optimize AI inference across a variety of chips, including those from Nvidia, AMD, and Google. This move positions ZML as a competitor to Baseten, recently valued at $13 billion.
TL;DR
- ZML releases LLMD, a free software tool for optimizing AI inference across multiple chip types, challenging Baseten's market position.
- The software aims to break vendor lock-in and reduce AI costs by enabling peak performance on various chips, including those from emerging European chipmakers.
- ZML's lean team of 20 has raised $20 million from notable investors, with plans for future releases and potential monetization.
What happened
ZML, a Paris-based AI startup, has launched LLMD, an inference server software that allows various open-source large language models to run on chips from different manufacturers, including Nvidia, AMD, Google's TPU, Apple Metal, and Intel Arc. The software is designed to maximize the speed of AI inference across these chips, potentially offering faster performance than native solutions.
ZML's founder, Steeve Morin, aims to disrupt the market by providing enterprises and cloud providers with the option to use a mix of chips, which could reduce costs and energy consumption. The software is currently free, with ZML planning to monetize it in the future based on usage data.
ZML has raised $20 million from venture firms and notable investors, including Harry Stebbings' 20VC, Kima Ventures, and Yann LeCun. The startup's lean team of 20 has enabled it to move quickly, with more releases planned.
Why it matters
LLMD's ability to optimize AI inference across multiple chip types challenges the market dominance of Nvidia and other established players, potentially reducing vendor lock-in and lowering AI costs. This could be particularly beneficial for enterprises and cloud providers looking to optimize their AI infrastructure.
The software also supports novel AI chipmakers, many of which are based in Europe. By enabling peak performance on a variety of chips, LLMD could accelerate the adoption of AI technologies and foster innovation in the semiconductor industry.
However, LLMD faces competition from other inference optimization tools, such as Baseten (valued at $13 billion), Inferact, and RadixArk. The long-term success of LLMD will depend on its ability to differentiate itself in a crowded market and demonstrate tangible benefits for users.
Key facts
- ZML's LLMD software is designed to optimize AI inference across chips from Nvidia, AMD, Google, Apple, and Intel.
- The software is currently free, with ZML planning to monetize it in the future based on usage data.
- ZML has raised $20 million from venture firms and notable investors, including Yann LeCun.
- The startup's lean team of 20 has enabled it to move quickly, with more releases planned.
- LLMD competes with other inference optimization tools, such as Baseten (valued at $13 billion), Inferact, and RadixArk.
- ZML aims to support novel AI chipmakers, many of which are based in Europe.
- The software is not open source, unlike ZML's first public project, an inference-focused ML framework released in 2024.
Context
The launch of LLMD comes amid a growing trend of investment in AI inference, often referred to as the 'inference gold rush'. As AI becomes more integrated into our work and everyday lives, optimizing inference has become increasingly important, often outpacing model training in terms of resource consumption and cost.
ZML's focus on multi-chip optimization positions it as a potential disruptor in the AI market, challenging the dominance of established players like Nvidia. By enabling peak performance on a variety of chips, LLMD could accelerate the adoption of AI technologies and foster innovation in the semiconductor industry.
However, the long-term success of LLMD will depend on its ability to differentiate itself in a crowded market and demonstrate tangible benefits for users. With competition from other inference optimization tools, such as Baseten, Inferact, and RadixArk, ZML will need to continue innovating and adapting to meet the evolving needs of the AI market.