LangSmith
LangSmith is a managed observability, evaluation, and testing platform for LLM applications, built by the team behind the LangChain framework.
Publisher review
LangSmith is a managed observability, evaluation, and testing platform for LLM applications, built by the team behind the LangChain framework. It is designed for teams that need to trace, monitor, and debug complex agentic systems, especially those built with LangChain or LangGraph. The platform provides complete visibility into agent behavior, from individual LLM calls to multi-step tool interactions, and is used by notable companies such as Klarna, Vanta, LinkedIn, Lyft, and Nvidia. LangSmith is best suited for teams already invested in the LangChain ecosystem who want a tightly integrated, production-ready solution for tracing and monitoring, rather than a framework-agnostic open-source alternative.
LangSmith organizes observability data into hierarchical 'Run Trees' composed of 'runs' or 'spans,' which capture the full lifecycle of a chain or agent execution. It offers pre-built and custom dashboards for traces, errors, token usage, cost, and tool latency, along with native alerting and webhooks for monitoring metrics. The platform supports both online and offline evaluations, including LLM-as-a-judge, gold standard evals, and human annotation queues. LangSmith also includes prompt management with versioning, a playground, and a 'Canvas' for prompt iteration. SDKs are available for Python, TypeScript, Go, and Java, allowing integration with any agent stack beyond LangChain.
LangSmith competes directly with Langfuse, Weights & Biases (W&B), Vellum, Galileo, and Fiddler AI. Compared to Langfuse, which is open-source and framework-agnostic with 17.9k+ GitHub stars, LangSmith is a closed-source, managed service with deep LangChain-native integration. LangSmith has 105M PyPI downloads versus Langfuse's 6.3M, reflecting its tight coupling with the LangChain library. While Langfuse offers strong OpenTelemetry support and a metrics API for custom alerts, LangSmith provides native alerting and webhooks out of the box. LangSmith's adoption is driven by its ecosystem lock-in, while Langfuse appeals to teams wanting transparency and vendor independence.
The honest trade-offs are clear. LangSmith's deep integration with LangChain and LangGraph is a strength for teams all-in on that stack, but it limits flexibility for those using other frameworks like LlamaIndex or Haystack. The free Developer plan includes only 5,000 traces per month with a 14-day data retention, which is restrictive for serious prototyping. The platform lacks pre-built tests or drift detection, requiring teams to build their own evaluation pipelines. Additionally, LangSmith is a managed service with no self-hosted option, which may be a concern for teams with strict data residency or compliance requirements.
How it works
-
Hierarchical run tracing
Captures agent behavior as 'Run Trees' of runs or spans, showing the full lifecycle of LLM calls, tool interactions, and sub-chains.
-
Pre-built and custom dashboards
Dashboards for traces, errors, token usage, cost, and tool latency, with custom metrics for monitoring production performance.
-
Native alerting and webhooks
Configurable alerts and webhooks for monitoring metrics, enabling teams to respond to anomalies in real time.
-
Online and offline evaluations
Supports LLM-as-a-judge, gold standard evals, and human annotation queues for both online and offline evaluation workflows.
-
Prompt management with versioning
Includes a 'Prompt Hub' with versioning, a playground for testing, and a 'Canvas' for iterative prompt development.
-
Multi-language SDK support
SDKs for Python, TypeScript, Go, and Java allow integration with any agent stack, not just LangChain.
-
Human-in-the-loop annotation
Annotation queues enable human reviewers to label and evaluate traces, supporting human-in-the-loop evaluation workflows.
Strengths and trade-offs
Strengths
- Immediate visual feedback on chains via the LangSmith UI, allowing teams to see exactly where a chain breaks during development.
- Deep, native integration with LangChain and LangGraph, providing the lowest friction for teams already using those frameworks.
- Pre-built dashboards for traces, errors, token usage, cost, and tool latency, with native alerting and webhooks for production monitoring.
- Adopted by top teams including Klarna, Vanta, LinkedIn, Lyft, and Nvidia, indicating strong enterprise trust and production validation.
Trade-offs
- Limited to the LangChain ecosystem for deep integration; teams using other frameworks like LlamaIndex or Haystack will find less native support.
- The free Developer plan includes only 5,000 traces per month with a 14-day data retention, which is restrictive for serious prototyping or small teams.
- No pre-built tests or drift detection, requiring teams to build custom evaluation pipelines for regression and data drift monitoring.
- Managed service only with no self-hosted option, which may be a concern for teams with strict data residency or compliance requirements.
Pricing context
Free Developer plan: 5,000 traces/month, 14-day retention. Plus plan: $39/seat/month, 10,000 traces/month, $0.50 per 1,000 extra traces, 400-day retention, unlimited seats, annotation queues, advanced monitoring, and online evaluations. Enterprise: custom pricing with SSO/SAML, RBAC, dedicated infrastructure, SLA, audit logs, custom retention, volume discounts, and priority support.
Getting started with LangSmith
-
Sign up for LangSmith
Go to the LangSmith website and create an account. Choose the Free Developer plan to start with 5,000 traces per month and 14-day retention, or select a paid plan for higher limits and longer retention.
-
Install the SDK
Install the LangSmith SDK for your language: run `pip install langsmith` for Python, or use the appropriate package manager for TypeScript, Go, or Java. This SDK enables tracing and evaluation from your application.
-
Connect your LangChain app
Configure your LangChain or LangGraph application to send traces to LangSmith by setting the `LANGCHAIN_TRACING_V2=true` environment variable and your API key. This automatically captures all runs and spans.
-
Run a trace and inspect it
Execute a chain or agent in your application. Open the LangSmith UI, navigate to the 'Traces' dashboard, and click on a run to see the hierarchical Run Tree, including LLM calls, tool interactions, and latency details.
-
Set up an alert for errors
In the LangSmith UI, go to 'Alerts' and create a new alert for error rate or token usage. Configure the threshold and add a webhook URL to receive notifications in your team's chat or monitoring system.
Frequently Asked Questions
What is LangSmith and what does it do?
LangSmith is a managed platform for tracing, monitoring, and evaluating LLM applications. Built by the LangChain team, it provides hierarchical run tracing, dashboards, alerting, and evaluation tools to debug and optimize agentic systems in production.
How does LangSmith compare to Langfuse?
LangSmith is a closed-source, managed service with deep LangChain integration, while Langfuse is open-source and framework-agnostic. LangSmith has 105M PyPI downloads versus Langfuse's 6.3M, but Langfuse offers stronger OpenTelemetry support and vendor independence.
What are the pricing plans for LangSmith?
LangSmith offers a free Developer plan with 5,000 traces per month and 14-day retention. The Plus plan costs $39 per seat per month, includes 10,000 traces, $0.50 per 1,000 extra traces, and 400-day retention. Enterprise has custom pricing with SSO and audit logs.
What are the main features of LangSmith?
Key features include hierarchical run tracing as Run Trees, pre-built dashboards for traces and costs, native alerting and webhooks, online and offline evaluations with LLM-as-a-judge, prompt management with versioning, and multi-language SDKs for Python, TypeScript, Go, and Java.
What are the limitations of LangSmith?
LangSmith's deep integration is limited to the LangChain ecosystem, so teams using LlamaIndex or Haystack get less native support. The free plan restricts traces to 5,000 per month with 14-day retention. It lacks pre-built tests and drift detection, and has no self-hosted option.
Which companies use LangSmith in production?
Notable companies using LangSmith include Klarna, Vanta, LinkedIn, Lyft, and Nvidia. These teams rely on LangSmith for production monitoring, tracing, and evaluation of their LLM applications, indicating strong enterprise trust and validation.
Alternatives
How LangSmith compares
Direct head-to-head against 3 competitors. Picked by 7wData.
LangSmith
- Pricing
- Free Developer plan: 5,000 traces/month, 14-day retention. Plus plan: $39/seat/month, 10,000 traces/month, $0.50 per 1,000 extra traces, 400-day retention, unlimited seats, annotation queues, advanced monitoring, and online evaluations. Enterprise: custom pricing with SSO/SAML, RBAC, dedicated infrastructure, SLA, audit logs, custom retention, volume discounts, and priority support.
- Target
- LangSmith is a managed observability, evaluation, and testing platform for LLM applications, built by the team behind the LangChain framework.
- Strength
- Immediate visual feedback on chains via the LangSmith UI, allowing teams to see exactly where a chain breaks during development.
- Watch for
- Limited to the LangChain ecosystem for deep integration; teams using other frameworks like LlamaIndex or Haystack will find less native support.
Braintrust
- Pricing
- $99/month base + $0.10/trace for production monitoring
- Target
- Teams needing CI/CD gates and controlled releases
- Deployment
- SaaS or self-host
- Strength
- Structured evaluations and deployment enforcement
- Watch for
- Pricing scales with trace volume
Langfuse
- Pricing
- Open-source or $0.0025/event for cloud
- Target
- Teams prioritizing open-source flexibility
- Deployment
- Self-host or cloud
- Strength
- Full trace visibility and prompt versioning
- Watch for
- Self-hosting requires infra management
Helicone
- Pricing
- Open-source or $0.0025/request for cloud
- Target
- Developers needing cost analytics
- Deployment
- Self-host or gateway
- Strength
- Real-time cost and latency tracking
- Watch for
- Limited built-in eval metrics
User reviews
No user reviews yet. Be the first to write one.
Sources
Reporting on this tool draws on these publicly available sources.