Comet ML (Opik)

Comet ML is a New York-based ML observability platform founded in 2017 that has pivoted toward GenAI workloads with Opik, an open-source LLM evaluation and tracing framework.

Reviewed by 7wData
API Available

On this page

Publisher review

Comet ML is a New York-based ML observability platform founded in 2017 that has pivoted toward GenAI workloads with Opik, an open-source LLM evaluation and tracing framework. The platform serves two audiences: teams running traditional ML experiments who want experiment tracking and model registry, and teams building LLM applications who need observability, prompt management, and LLM-as-a-judge evaluation. The MLOps suite is Comet's core product for classical ML workflows, but Opik—released September 2024—is the company's bet on LLM observability as a faster, leaner alternative to Weights & Biases and LangSmith.

Opik logs traces at 7× the speed of Langfuse and 14× the speed of Arize Phoenix in Comet's benchmarks, and the open-source version includes production-ready features like online evaluation rules and 30+ LLM-as-a-judge metrics. However, Opik is the newest player in a crowded space; production deployments require Kubernetes, and self-hosted setups lack built-in authentication. Integration paths exist for LangChain, liteLLM, and LangGraph, with automatic cost tracking limited to OpenAI and Google.

The company raised $63M and employs ~102 as of early 2026. For teams already on Comet's ML platform, Opik adds GenAI observability without vendor switching; for new LLM projects, Langfuse remains the stronger open-source starting point due to broader framework coverage and lower operational overhead.

Get the AI & data signal, daily.

335k+ subscribers read this every morning. One email, both newsletters. Unsubscribe anytime.

How it works

  1. Trace logging and span recording

    Capture full execution traces of LLM calls, tool invocations, and agent steps with automatic input/output logging and metadata tagging for multi-turn conversations.

  2. LLM-as-a-judge evaluation

    Auto-score production traces with 30+ built-in metrics (answer relevance, hallucination detection, context precision) without manual annotation overhead.

  3. Prompt versioning and A/B testing

    Centralized prompt library with structured experiment tracking to compare prompt variants, temperature settings, and chain-of-thought strategies side-by-side.

  4. Online evaluation rules

    Define evaluation conditions that automatically score a subset of live production traces, triggering alerts when quality degrades below thresholds.

  5. Human-in-the-loop review

    Invite subject matter experts to label and score traces directly in the platform; aggregate multiple reviewer scores to reduce individual bias and build training datasets.

  6. Experiment comparison dashboard

    Side-by-side analysis of experiment runs with configurable metrics, trace replay, and cost breakdowns per feature and model.

  7. Self-hosted and managed deployment options

    Run Opik locally via Docker Compose for development, or deploy on Kubernetes for production; full open-source codebase under Apache 2.0 license.

Strengths and trade-offs

Strengths

  • Permissive open-source licensing (Apache 2.0) and full self-hosting capability without Enterprise tier restrictions
  • 7–14× faster trace logging and evaluation than Langfuse and Arize Phoenix in published benchmarks
  • Unified platform for both classical ML tracking (via MLOps suite) and LLM observability (via Opik) if migrating from W&B

Trade-offs

  • Opik launched September 2024; limited production case studies and smaller community (19.4k GitHub stars) compared to Langfuse and LangSmith
  • Self-hosted deployments require Kubernetes for production and lack built-in authentication, necessitating external reverse proxy or network isolation
  • Cost tracking only automatic for OpenAI and Google; manual calculation required for Anthropic, Bedrock, and other providers despite token counts being available

Pricing context

Comet offers a three-tier model for Opik: Free Cloud ($0, limited to 10 team members, 25k spans/month, 60-day retention); Pro Cloud ($19/month, up to 50 team members, 100k spans/month, 60-day retention); Enterprise (custom pricing, unlimited users, flexible deployments, SSO). The open-source version is free and fully functional for local development or Kubernetes self-hosting. For the MLOps suite, Free ($0 for 1 user with experiment tracking and 100GB storage) and Pro ($19/user/month for up to 10 users with 1,500 training hours and 500GB storage).

Academic users receive the Pro plan at no cost. All tiers include LLM evaluation metrics for free.

Getting started with Comet ML (Opik)

  1. Sign up for Opik Cloud

    Go to comet.com and create a free account. Choose the Free Cloud tier to start with 10 team members, 25k spans per month, and 60-day retention. No credit card is required for the free plan.

  2. Install the Opik SDK

    Run `pip install opik` in your Python environment. For local development, also install Docker Compose to run the open-source version. The SDK provides the client library for logging traces and evaluations.

  3. Configure API key and project

    Set your Comet API key as an environment variable: `export COMET_API_KEY=<your_key>`. Then initialize Opik in your code with `import opik` and create a project using `opik.init(project_name="my-llm-app")`.

  4. Log your first LLM trace

    Wrap an LLM call with Opik's tracing: use `@opik.track` decorator on a function that calls OpenAI or Google models. The trace automatically captures inputs, outputs, latency, and token usage. View the trace in the Opik dashboard.

  5. Set up an online evaluation rule

    In the Opik UI, navigate to Evaluations and create a new online rule. Choose a built-in metric like answer relevance, set a threshold, and apply it to a subset of production traces. Enable alerts to notify your team when quality drops.

Frequently Asked Questions

What is Comet ML Opik and what does it do?

Opik is an open-source LLM evaluation and tracing framework from Comet ML, released in September 2024. It captures full execution traces of LLM calls, supports online evaluation rules, and includes 30+ built-in LLM-as-a-judge metrics for automated scoring.

How much does Opik cost and what are the pricing tiers?

Opik offers a Free Cloud tier with 10 team members and 25,000 spans per month, a Pro Cloud tier at $19 per month for up to 50 members and 100,000 spans, and Enterprise custom pricing. The open-source version is free for self-hosting.

How does Opik compare to Langfuse in terms of speed and features?

Opik logs traces 7 times faster than Langfuse and 14 times faster than Arize Phoenix in Comet's benchmarks. However, Langfuse has broader framework coverage, a larger community, and lower operational overhead for new LLM projects.

What are the main features of Opik for LLM observability?

Key features include trace logging and span recording, LLM-as-a-judge evaluation with 30+ metrics, prompt versioning and A/B testing, online evaluation rules for live production traces, and human-in-the-loop review for labeling and scoring.

Can Opik be self-hosted and what are the deployment requirements?

Yes, Opik is fully open-source under Apache 2.0 and can be self-hosted. For local development, use Docker Compose. Production deployments require Kubernetes, and self-hosted setups lack built-in authentication, needing an external reverse proxy.

What are the weaknesses of Opik for production use?

Opik launched in September 2024, so it has limited production case studies and a smaller community compared to Langfuse. Self-hosted deployments require Kubernetes and lack built-in authentication. Automatic cost tracking only works for OpenAI and Google models.

Alternatives

How Comet ML (Opik) compares

Direct head-to-head against 2 competitors. Picked by 7wData.

This tool

Comet ML (Opik)

Pricing
Comet offers a three-tier model for Opik: Free Cloud ($0, limited to 10 team members, 25k spans/month, 60-day retention); Pro Cloud ($19/month, up to 50 team members, 100k spans/month, 60-day retention); Enterprise (custom pricing, unlimited users, flexible deployments, SSO). The open-source version is free and fully functional for local development or Kubernetes self-hosting. For the MLOps suite, Free ($0 for 1 user with experiment tracking and 100GB storage) and Pro ($19/user/month for up to 10 users with 1,500 training hours and 500GB storage). Academic users receive the Pro plan at no cost. All tiers include LLM evaluation metrics for free.
Target
Comet ML is a New York-based ML observability platform founded in 2017 that has pivoted toward GenAI workloads with Opik, an open-source LLM evaluation and
Strength
Permissive open-source licensing (Apache 2.0) and full self-hosting capability without Enterprise tier restrictions
Watch for
Opik launched September 2024; limited production case studies and smaller community (19.4k GitHub stars) compared to Langfuse and LangSmith

Langfuse

Pricing
Free tier; Team $59/user/month; Enterprise custom
Target
Developers needing structured prompt management and LLM observability
Deployment
Open source self-host or cloud
Strength
Built-in prompt management and customizable dashboards for LLM observability
Watch for
No multi-modal evaluation support; evaluation dashboard is partial

Neptune.ai

Pricing
Free tier; Pro $199/user/month; Enterprise custom
Target
ML teams needing experiment tracking and model registry
Deployment
SaaS only
Strength
Mature experiment tracking and model registry with rich metadata logging
Watch for
Pricing escalates quickly with team size; no native LLM evaluation features

User reviews

No user reviews yet. Be the first to write one.

Sources

Reporting on this tool draws on these publicly available sources.

  1. www.comet.com — Opik and MLOps pricing tiers, feature limits, and team size caps for Free, Pro, and Enterprise plans
  2. github.com — Open-source codebase, GitHub stars (19.4k), active community engagement, and framework integrations
  3. www.comet.com — Opik announcement date (September 2024) and core feature set for LLM evaluation and observability
  4. www.comet.com — Opik's speed benchmarks (7–14× faster than competitors), comparison with Ragas and other frameworks
  5. bigdataboutique.com — Trade-offs between Opik, Langfuse, and LangSmith; weaknesses regarding production maturity and community size
  6. medium.com — Real-world implementation insights including cost tracking limitations, security gaps in self-hosted versions, and evaluation learning curve
  7. tracxn.com — Comet founding year (2017), headquarters (New York), Series B status, $63M funding, and employee count (102 as of 2026)
  8. docs.litellm.ai — liteLLM and Opik integration capabilities for automatic logging in LLM gateway setups