DataTalks.Club

Articles Slack Events Podcast Wiki Books Courses

Course Wiki

Embeddings — LLM Zoomcamp Module 2

--- video_url: 'https://www.youtube.com/watch?v=kJOlW1HeMp4&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv' --- # Embeddings

Related Pages

llmz-module-02

LLM Zoomcamp › Module 2: Vector Search › Embeddings

Notes

video_url: “https://www.youtube.com/watch?v=kJOlW1HeMp4&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv”

Embeddings

Before we can do vector search, we need to turn our text into vectors. We call this process embedding: we embed text into a vector space. The vectors we get back are also called “embeddings.”

Word embeddings and sentence embeddings

This idea comes from word2vec. The model learns to place words as points in a multi-dimensional space. Words with similar meanings land close to each other.

Imagine a 2D space where “enroll” and “join” are near each other and “Docker” is far away:

        · enroll
       · join
                   · Docker

The same idea works for entire sentences:

Q1: "I just discovered the course. Can I still join it?"
Q2: "I just found out about the program. Can I still enroll?"

These two are close - they mean the same thing.

Q3: "How do I run Docker on Windows?"

This one is far away from Q1 and Q2.

Now imagine all 1200 documents in our FAQ dataset. Each one becomes a point in this space. When a user asks a question, we embed it into the same space and find the closest documents. Those nearest neighbors are our search results.

The model encodes the whole sentence, not the words in isolation. So it can tell apart the same word in different contexts.

Take the word “judge.” In “the judge ruled out the possibility of crime” (legal) it gets one vector. In “LLM-as-a-judge approach to evaluate LLMs” (ML evaluation) it gets a different one. The surrounding context changes the embedding.

So an embedding model takes text in and returns a fixed-length array of numbers. We train it so that texts with similar meanings get similar vectors.

We’ll use sentence-transformers, a popular open-source library for embeddings. It runs locally on your machine, so there are no API costs.

Install sentence-transformers

Recent versions of sentence-transformers install the GPU-enabled PyTorch package, which pulls large NVIDIA CUDA libraries. It takes a few gigabytes of disk space, even if you don’t have a GPU.

To avoid this, you need to explicitly tell uv to install the CPU-only PyTorch wheels.

Add a CPU-only PyTorch index to pyproject.toml:

[tool.uv.sources]
torch = { index = "pytorch-cpu" }

[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"

Install the library:

uv add sentence-transformers

If your pyproject.toml already specifies the requests package, make sure it is pinned to requests>=2.28.1, because pytorch-cpu requires this version to install correctly.

In this way, uv will automatically resolve PyTorch using the CPU-only index.

We will also see how to trim it down even more for using in production in the ONNX Embedder lesson later.

Choosing a model

Sentence-transformers supports many models. The right one depends on your task, your language, and the resources you have. Larger models are usually slower, so for our FAQ dataset of short English texts a small model is enough. Try a few on your own data and keep the one that works best.

We’ll use all-MiniLM-L6-v2:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")

The first time you run this, it downloads the model (~80 MB) and the tokenizer from HuggingFace. The tokenizer turns text into something the model can read. After that, both load from a local cache.

Trying it with simple examples

Let’s see how embeddings work on a few examples.

We’ll start with a query:

q1 = "Can I still join the course after the start date?"
v1 = model.encode(q1)

v1 is a vector, an array of 384 numbers. Each number stands for some concept the model learned. We can’t read off what a

Key concepts

Sources

DataTalks.Club. Hosted on GitHub Pages. Built with Rustkyll. We use cookies.