Retries, Fallbacks and Errors

Real apps break. The internet drops. The provider is busy. Your key runs out of credit.

A good app does not crash when that happens. It tries again, or switches to a backup, or says sorry nicely.

A fallback is a spare tyre. You hope you never need it, but you are very glad it is in the boot.

with_retry: try again

Many errors are temporary. Waiting a second and trying again often fixes them.

with_retry wraps the model. If a call fails, it waits a little and tries again, up to 3 times.

It works on any Runnable, not just models. You can retry a whole chain.

with_fallbacks: the spare tyre

If one model is down, use another one. The user never notices.

LangChain calls main_model first. If it raises an error, it calls backup_model with the same input.

You can give a list of several backups. They are tried in order.

Fallbacks for a whole chain

Sometimes the backup needs a different prompt too. Then fall back at the chain level.

The whole main_chain is tried first. If any step fails, the whole backup_chain runs instead.

try and except: catch what is left

Retries and fallbacks handle most trouble. For everything else, wrap the call in try and except.

Specific errors come first, so you can give a specific message. The plain Exception at the end catches anything you did not expect.

Timeouts and token caps

A call that never returns is worse than an error. Give it a deadline.

And cap how long the answer can be, so one runaway reply does not eat your budget.

timeout raises APITimeoutError when the clock runs out. max_tokens keeps every answer short and cheap.

A graceful step inside a chain

You can put the try and except inside a RunnableLambda. Then the chain itself never crashes.

Whatever happens, the user gets a sentence back, not a red error screen.

Putting the layers together

Retry the main model twice. If it still fails, switch to the backup. That is a solid default for most apps.

Remember: errors are normal. Plan for them on day one, and your app will feel calm and trustworthy.