en.Wedoany.com Reported - Baseten Inc. is raising $1.5 billion in funding, with the round co-led by Altimeter Capital, Conviction, Spark Capital, Sands Capital, and Wellington Management. According to The Wall Street Journal, some investors purchased shares at a valuation of $11 billion, while the term sheets for other backers specified a valuation of $13 billion.
Baseten offers a service platform for automating the setup and management of cloud inference clusters. Through this platform, developers can eliminate tedious tasks such as allocating graphics cards, configuring networking, and installing software tools. Baseten's software can be used both as a managed service and as a standalone application deployed in a company's public cloud environment.
The core of the platform consists of three modules known as inference engines, which optimize the performance of customer AI models and collect technical issue data. The BIS-LLM engine is designed for large language models that adopt a mixture-of-experts architecture. Such models contain multiple neural networks, each tailored for different tasks. BIS-LLM enhances model efficiency by optimizing its KV cache and automatically provisions more hardware as the model's token usage increases. The second engine, Engine-Builder-LLM, is optimized for dense LLMs, which consist of a single collection of artificial neurons rather than multiple neural networks. Engine-Builder-LLM employs speculative decoding technology to generate multiple tokens at once, thereby accelerating inference. The third engine, BEI, targets simpler AI models and can power embedding models, data classification, and search models, among others.
Baseten uses a software module called MCM to distribute inference workloads across multiple public clouds. If one cloud experiences a failure, MCM reroutes prompts to platforms that remain online. The company states that the technology's ability to switch providers is also convenient during shortages of graphics cards in major public clouds. The platform directly supports dozens of open-source AI models, and customers can deploy custom algorithms using a tool called Truss, which automatically packages LLMs into a Baseten-compatible format.
The Baseten platform not only supports inference on custom LLMs but also enables their training. The platform includes a backup feature that periodically saves copies of the neural network during training. If technical issues arise, developers can restore the latest backup copy without having to start training from scratch. This funding round comes less than six months after the previous one, a $300 million investment that included NVIDIA Corporation and CapitalG.
This article is compiled by Wedoany. All AI citations must indicate the source as "Wedoany". If there is any infringement or other issues, please notify us promptly, and we will modify or delete it accordingly. Email: news@wedoany.com
