Tracing with LangSmith
When a chain gives a strange answer, you want to see what happened inside.
Which prompt was really sent? What did the retriever fetch? How long did each step take?
LangSmith records all of that. It is a flight recorder for your AI app.
It is optional, made by the LangChain team, and has a free tier that is plenty for learning.
Step 1: Install and get a key
Sign up at smith.langchain.com and create an API key in settings.
Step 2: Three environment variables
Add these to your .env file next to your OpenAI key.
LANGSMITH_TRACING=true switches recording on. LANGSMITH_PROJECT is the folder your runs are filed under.
That is the whole setup. You do not change a single line of Python.
Step 3: Run any chain
Now open smith.langchain.com and click your project. The run is already there.
What you see in a trace
The input you gave and the final output
Every step: prompt, model, parser, each with its own timing
The exact messages sent to the model, after the template was filled in
Token counts and the cost of the call
Any error, with the full stack trace
For a RAG chain you also see the exact chunks the retriever picked. This is the fastest way to find out why an answer was wrong.
Give runs a name
Many runs with the same name are hard to tell apart. Give them a clear label.
run_name shows up as the title of the trace, so you can filter and search for it later.
Trace your own functions
Plain Python functions are not Runnables, so LangSmith does not see them by default.
Add the @traceable decorator and they show up as steps too.
Now the trace shows ask, then clean inside it, then the chain inside that. One tree, top to bottom.
No account? Debug offline
If you do not want a cloud service, LangChain can print the same details to your terminal.
It is noisy, but you see every input and output of every step. Turn it off when you are done.
When to use it
A chain gives a weird answer and you do not know why
Your app is slow and you want to find the slow step
You want to know what a day of usage costs
You are about to show your app to other people
Tip: Turn tracing on while you build, and leave it on in production. The one time something goes wrong, the recording pays for itself.
Test yourself
Three quick questions made just for this lesson. Earn 10 XP per correct answer.