DataTalks.Club

Articles Slack Events Podcast Wiki Books Courses

Course Wiki

The LLM — LLM Zoomcamp Module 1

--- video_url: 'https://www.youtube.com/watch?v=KHePGkeFn54&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv' code: - label: 'notebook.ipynb' path: 'code/notebook.ipynb' --- # The LLM

Related Pages

llmz-module-01

LLM Zoomcamp › Module 1: Agentic RAG › The LLM

Notes

video_url: “https://www.youtube.com/watch?v=KHePGkeFn54&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv” code:

# The LLM

The last component of our RAG pipeline is the LLM. It takes the prompt we built and generates an answer.

Sending the prompt to the LLM

We have the prompt from the previous section.

We send it to the LLM:

response = openai_client.responses.create(
    model="gpt-5.4-mini",
    input=prompt
)

We use OpenAI’s Responses API (openai_client.responses.create). OpenAI has two APIs: chat completions and responses. Chat completions is the older one, and it’s now considered legacy. When the first edition of this course started, the Responses API didn’t exist, so we used chat completions. Now we prefer responses because it’s more convenient.

There’s a catch worth knowing. Many other providers like Groq and Gemini give you an OpenAI-compatible client. But they expose chat completions, not responses. So if you switch providers, you keep the OpenAI client but call chat.completions instead of responses.

Exploring the response

The response is a Pydantic object. The answer is in response.output - a list of output items.

The first one is the message:

response.output[0]

The message has a content list, and the text is in the first item:

response.output[0].content[0].text

That’s quite a journey to reach one string.

The shortcut spares us all of it:

response.output_text

Same result, less code. The answer should be something like: “Yes, you can still join. If you want to receive a certificate, make sure to submit your project while submissions are still open.”

The usage counts tell you how many tokens the request consumed:

response.usage

You’ll see something like:

ResponseUsage(input_tokens=334, output_tokens=39, total_tokens=373)

Calculating the price

You can use different models.

In this course we’ll use gpt-5.4-mini:

Let’s calculate the cost of the request we just made:

input_price = 0.75 / 1_000_000
output_price = 4.50 / 1_000_000
cost = (
    response.usage.input_tokens * input_price +
    response.usage.output_tokens * output_price
)
cost

This particular request costs a fraction of a cent. Even a full RAG query with a long prompt stays under $0.01. We need to send a lot of queries to even spend one cent. These models are cheap to play with.

The usage object also reports cached input tokens. Those are billed at a lower rate when the same prompt prefix repeats.

Message history

Previously we sent only one string as input and got back a response. In practice, you typically send a message history - a list of messages where each message has a role.

Think of a ChatGPT conversation. It starts with a hidden system prompt that tells the LLM how to behave, one you never see. After that, your messages and the LLM’s replies alternate. The LLM has no memory of its own, so it needs the full history passed in to continue the conversation.

We won’t build a multi-turn chat here. But we still use this message format to separate our instructions from the user prompt.

We send two messages:

message_history = [
    {"role": "developer", "content": INSTRUCTIONS},
    {"role": "user", "content": prompt}
]
response = openai_client.responses.create(
    model="gpt-5.4-mini",
    input=message_history
)

This separates the fixed instructions from the user prompt, which changes every request.

OpenAI accepts both developer and system for the instruction role. There may be some difference between them, but in practice I don’t see it change the result either way. We use

Key concepts

Sources

DataTalks.Club. Hosted on GitHub Pages. Built with Rustkyll. We use cookies.