LangSmith is an all-in-one developer platform for every step of the LLM-powered application lifecycle, whether you’re building with LangChain or not.
Debug, collaborate, test, and monitor your LLM applications.
LLM-apps are powerful, but have peculiar characteristics. The non-determinism, coupled with unpredictable, natural language inputs, make for countless ways the system can fall short. Traditional engineering best practices need to be re-imagined for working with LLMs, and LangSmith supports all phases of the development lifecycle.
Unexpected results happen all the time with LLMs. With full visibility into the entire sequence of calls, you can spot the source of errors and performance bottlenecks in real-time with surgical precision. Debug. Experiment. Observe. Repeat. Until you’re happy with your results.
Building LLM-powered applications requires a close partnership between developers and subject matter experts.
Traces
Easily share a chain trace with colleagues, clients, or end users, bringing explainability to anyone with the shared link.
Hub
Use LangSmith Hub to craft, version, and comment on prompts. No engineering experience required.
Annotation Queues
Try out LangSmith Annotation Queues to add human labels and feedback on traces.
Datasets
Easily collect examples, and construct datasets from production data or existing sources. Datasets can be used for evaluations, few-shot prompting, and even fine-tuning.
Layer in human feedback on runs or use AI-assisted evaluation,
with off-the-shelf and custom evaluators that can check for relevance, correctness, harmfulness, insensitivity, and more.
Quickly save debugging and production traces to datasets. Datasets are collections of either exemplary or problematic inputs and outputs that should be replicated or rectified, respectively.
Use an LLM and prompt to score your application output, or write your own functional evaluation tests to record different measures of effectiveness.
See how performance of the evaluation criteria that you’ve defined is affected by changes to your application. Know that quality is moving in a positive direction by applying engineering rigor to your testing workflow.
Continuously track qualitative characteristics of any live application, and spot issues in real-time with LangSmith monitoring.
See what’s happening with your production application, so you can take action when needed or rest assured while your chains and agents do the hard work.
Yes! Many companies who don’t build with LangChain use LangSmith. You can log traces to LangSmith via the Python SDK, the TypeScript SDK, or the API. See here for more information.
Yes, we allow customers to self-host LangSmith on our enterprise plan. We deliver the software to run on your Kubernetes cluster, and data will not leave your environment. For more information, check out our documentation.
Getting started is as easy as setting three environment variables in your LangChain code. When you use the LangSmith SDK, there’s a callback handler to collect traces and send them to your LangSmith Organization. All your trace steps will be formatted automatically, so there’s virtually no set up cost.
Yes, we have an API that allows you to programmatically interact with every feature LangSmith has to offer, including logging traces, creating datasets, and running evals. See documentation for more detail.
Traces are stored in GCP us-central-1. Organizations' traces are logically separated from each other in a Clickhouse database and encrypted in transit and at rest.
LangSmith traces contain the full information of all the inputs and outputs of each step of the application. This level of information is needed, so that users have full visibility into what's happening in their applications for debugging purposes.
Yes, starting with the LangSmith Python SDK version 0.0.84 and JS SDK version 0.0.64, you can specify the % of traces you send to LangSmith. See documentation for more detail.
No, LangSmith does not add any latency to your application. In the LangSmith SDK, there’s a callback handler that sends traces to a LangSmith trace collector which runs as an async, distributed process. Additionally, if LangSmith experiences an incident, your application performance will not be disrupted.
We will not train on your data, and you own all rights to your data. See LangSmith Terms of Service for more information.
See our pricing page for more information, and find a plan that works for you.
Get started with LangChain, LangSmith, and LangGraph to enhance your LLM app development, from prototype to production.