vLLM
vLLM is an open-source inference engine for large language models, developed at UC Berkeley's Sky Computing Lab under the supervision of Ion Stoica, a co-founder of Databricks.
Profile
vLLM is an open-source software library that speeds up and reduces the cost of running large language models (LLMs) for inference, enabling companies to deploy AI chatbots and other generative AI applications more efficiently.
vLLM is an open-source inference engine for large language models, developed at UC Berkeley's Sky Computing Lab under the supervision of Ion Stoica, a co-founder of Databricks. The project, launched in 2023, quickly became one of the most popular open-source tools for LLM serving, accumulating over 82,000 GitHub stars and 17,000 forks by June 2026. In January 2026, the core team spun out the project into a VC-backed startup called Inferact, raising $150 million in seed funding at an $800 million valuation, co-led by Andreessen Horowitz and Lightspeed Venture Partners.
Inferact CEO Simon Mo, one of the original creators, stated that existing users include Amazon Web Services and a major shopping app. The company's primary product remains the vLLM inference engine, which competes with other open-source projects like SGLang (spun out as RadixArk at a $400 million valuation) and commercial offerings from NVIDIA and others. As of June 2026, vLLM has not disclosed its own revenue or headcount; the project is funded through GitHub Sponsors and Open Collective donations.
The broader inference market is booming, with NVIDIA reporting record Data Center revenue of $62.3 billion in Q4 FY2026, up 75% year-over-year, driven by demand for AI inference. Inferact's large seed round reflects investor appetite for infrastructure that reduces the cost of running AI models, but the company faces challenges in monetizing an open-source project and competing with well-funded rivals.
Products by vLLM
Who buys this
- Cloud service providers (e.g., Amazon Web Services) offering managed AI inference to their customers
- Large enterprises deploying internal AI chatbots and virtual assistants
- AI startups and developers building applications on top of LLMs
- Research institutions and universities running experiments on large language models
- Companies running on-premises or hybrid AI infrastructure for data privacy reasons
Publicly disclosed clients
- Amazon Web Services (AWS)
- A major shopping app (unnamed, per Bloomberg interview with CEO Simon Mo)
Strengths and what to watch
Strengths
- Massive open-source adoption: 82,000 GitHub stars and 17,322 commits as of June 2026, indicating a large and active developer community
- Strong academic pedigree: incubated at UC Berkeley's Sky Computing Lab under Ion Stoica, lending credibility and attracting top talent
- Substantial venture backing: $150 million seed round at an $800 million valuation from top-tier investors (a16z, Lightspeed) provides a long runway for commercialization
Watch for
- Monetization risk: vLLM is open-source and free; Inferact must convert a large user base into paying customers without alienating the community
- Competitive pressure: Rival open-source inference engines like SGLang (now RadixArk, valued at $400M) and commercial offerings from NVIDIA (which reported $62.3B in quarterly Data Center revenue) could erode market share
- Key-person dependency: The project's success is tied to a small core team; any departure of founders or lead maintainers could slow development and adoption
Recent moves
Key Information
- Industry
- MLOps & AI Infra
- Founded
- 2023
Frequently Asked Questions
What is vLLM and what does it do?
vLLM is an open-source software library that speeds up and reduces the cost of running large language models for inference. It helps companies deploy AI chatbots and other generative AI applications more efficiently by optimizing model serving.
Who created vLLM and where did it come from?
vLLM was developed at UC Berkeley's Sky Computing Lab under Ion Stoica, a co-founder of Databricks. The project launched in 2023 and quickly became a popular open-source tool for LLM serving, accumulating over 82,000 GitHub stars by June 2026.
Is vLLM free to use and how is it funded?
Yes, vLLM is open-source and free to use. The project is funded through GitHub Sponsors and Open Collective donations. In January 2026, the core team spun out Inferact, a VC-backed startup that raised $150 million to commercialize vLLM.
What companies use vLLM for AI inference?
Notable users include Amazon Web Services and a major shopping app, as stated by Inferact CEO Simon Mo. Cloud service providers, large enterprises deploying internal AI chatbots, AI startups, and research institutions also use vLLM for running large language models.
How does vLLM compare to other inference engines like SGLang?
vLLM competes with open-source projects like SGLang, which spun out as RadixArk at a $400 million valuation. Both aim to optimize LLM inference, but vLLM has a larger GitHub community with 82,000 stars and strong backing from top-tier investors like a16z.
What are the main challenges facing vLLM and Inferact?
Key challenges include monetizing an open-source project without alienating the community, competing with well-funded rivals like NVIDIA and RadixArk, and retaining the small core team that drives development. Inferact must convert its large user base into paying customers.
Sources
- github.com — GitHub repository statistics: 82,000 stars, 17,700 forks, 17,322 commits, and project structure
- techcrunch.com — Inferact's $150M seed round at $800M valuation, co-led by a16z and Lightspeed, and mention of AWS as a user
- techcrunch.com — Context on competitive landscape: SGLang spin-out RadixArk at $400M valuation
- nvidianews.nvidia.com — NVIDIA's Q4 FY2026 Data Center revenue of $62.3B, up 75% YoY, indicating market demand for inference