Inference

Weights & Biases (W&B) Inference is a managed inference service for open-source large language models (LLMs), providing both a REST API and an interactive Playground for developers to query models like Meta Llama 3.1 8B, DeepSeek V4-Pro, and Qwen3 235B A22B without provisioning GPU infrastructure. It is part of the broader W&B MLOps platform, which includes experiment tracking, a model registry with lineage, CI/CD automations, and LLM evaluation tools (Weave). The service targets individual developers, professional teams, and enterprises that need to deploy and monitor LLMs quickly while staying within the W&B ecosystem for experiment management and observability. W&B Inference is powered by CoreWeave GPU cloud infrastructure, and its pricing is entirely separate from W&B's experiment tracking plans (Free, Pro, Enterprise).

W&B Inference offers per-model, per-token pricing for hosted models, with costs varying by model size and capability. For example, input tokens for Meta Llama 3.1 8B cost $0.22 per million tokens, while output tokens for DeepSeek V4-Pro cost $3.46 per million tokens. Cache-hit discounts reduce costs significantly—Qwen3.6 27B input tokens drop from $0.60 to $0.12 per million tokens on cache hit. The service supports over 20 open-source models, including NVIDIA Nemotron 3 Super 120B, Google Gemma 4 31B, and Microsoft Phi 4 Mini 3.8B. Usage is billed monthly, and Playground queries are charged identically to API calls. W&B Inference does not include any free tier; users must have a paid W&B account (Pro or Enterprise) to access the API, and inference costs are added on top of the platform subscription.

W&B Inference competes directly with serverless inference providers such as Together AI, Fireworks AI, and Replicate, as well as cloud-native services like AWS Bedrock and GCP Vertex AI. It differentiates by tightly coupling inference with W&B's experiment tracking and model registry, enabling teams to log prompt-response pairs, evaluate outputs, and manage model versions in one place. However, it lacks the breadth of MLOps orchestration found in competitors like ZenML or MLflow, and its per-token pricing can be expensive for high-volume workloads compared to dedicated GPU endpoints from providers like CoreWeave or Lambda Labs. The service is not available on the free or Personal W&B plans, limiting its accessibility for individual developers.

The honest trade-offs: W&B Inference is convenient for teams already invested in the W&B platform, but its costs can escalate quickly for production-scale inference. A team running 10 million input tokens per month on DeepSeek V4-Pro would pay $17.40 for input tokens alone, plus $34.60 for output tokens, totaling $52.00 per month—before any W&B subscription fees. The service lacks support for custom model fine-tuning or deployment of private models, meaning users must rely on the provided open-source model roster. For budget-conscious teams, alternatives like self-hosted vLLM or Ollama offer zero per-token costs but require infrastructure management. W&B Inference is best suited for teams that prioritize rapid prototyping and integrated observability over raw cost efficiency.

W&B Inference is billed separately from W&B platform subscriptions. Per-model per-token pricing ranges from $0.05/M input tokens (JetBrains Mellum2 12B) to $1.74/M input tokens (DeepSeek V4-Pro). Cache-hit discounts reduce input costs by 50-80%.

Access requires a paid W&B Pro plan ($60/month) or Enterprise plan (custom pricing). No free tier for inference.