Streaming Answers

When you call invoke, you wait for the whole answer. Then it appears all at once.

Streaming sends the answer piece by piece, as the model writes it.

Users see words appear right away. The wait feels much shorter, even though the total time is the same.

It is like a friend reading a story out loud instead of handing you the book at the end.

stream on a model

stream returns a generator. You loop over it and get small chunks.

Each chunk is an AIMessageChunk. Its .content holds a few characters.

end='' stops print from adding a new line. flush=True pushes the text to the screen right away.

stream on a chain

Chains stream too. With a StrOutputParser at the end, each chunk is already a plain string.

No .content needed. The parser turns every chunk into a string as it passes through.

This works because the parser knows how to handle chunks, not just full messages.

Adding chunks together

Chunks can be added. That is how you rebuild the full message while streaming.

The plus sign glues two chunks into a bigger chunk. At the end you have everything.

Async streaming

Web servers and bots usually run async code. Every Runnable has an async twin with an a in front.

astream is the async version of stream. You also get ainvoke and abatch.

While one answer streams, your program can handle other users at the same time.

astream_events: watching every step

Sometimes you want more than the final text. You want to know when each step starts and ends.

Each event has a name and some data. You pick the ones you care about.

This is handy for showing a spinner while a tool runs, or logging what happened.

When to stream

Remember: swap invoke for stream and loop over the chunks. With a StrOutputParser, each chunk is already a string.