Free Local Models with Ollama

Ollama runs AI models on your own computer.

No API key. No bill. No internet needed once the model is downloaded.

It is like owning a small library at home instead of paying for every visit to the big one downtown.

Everything you learned so far works with it. You only change one line.

Step 1: Install Ollama and pull a model

Go to ollama.com and install it like any other app. Then open a terminal.

llama3.2 is a small chat model. nomic-embed-text makes embeddings for vector stores.

Step 2: Install the LangChain package

This is the bridge between LangChain and the Ollama app running on your machine.

Step 3: Use it as a chat model

Or use the same init_chat_model front door you already know.

Same invoke. Same AIMessage. Same .content. Nothing else changes.

Swap the model in a real chain

Here is a chain from earlier lessons. Only the model line is different.

Prompts, parsers, streaming and pipes are all the same. That is the whole point of LangChain.

Free embeddings too

RAG needs embeddings. Ollama can make those for free as well.

Now your whole RAG pipeline can run with zero cloud calls.

Ollama on another machine

By default LangChain talks to http://localhost:11434. If Ollama runs on a different computer, point to it.

This is handy when one strong computer at home serves models to your laptop.

Strengths and weaknesses

A good habit is to build with a local model for free, then switch to a cloud model for the final version if you need more quality.

The Fox AI helper on this website can also run on Ollama. Same idea, same code.

Tip: If Ollama answers feel slow, try a smaller model like llama3.2:1b, or close other heavy apps to free up memory.