Course Wiki

Building the Prompt — LLM Zoomcamp Module 1

--- video_url: 'https://www.youtube.com/watch?v=DV4e2n-dIv0&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv' code: - label: 'notebook.ipynb' path: 'code/notebook.ipynb' --- # Building the Prompt

llmz-module-01

LLM Zoomcamp › Module 1: Agentic RAG › Building the Prompt

Notes

video_url: “https://www.youtube.com/watch?v=DV4e2n-dIv0&list=PL3MmuxUbc_hLZFNgSad56pDBKK8KO0XIv” code:

label: “notebook.ipynb” path: “code/notebook.ipynb”

# Building the Prompt

The LLM doesn’t see our documents unless we pass them in. So we need to build a prompt that includes the user’s question and the search results.

When we build AI systems, we usually split the prompt into two parts:

Instructions (also called the system prompt): this tells the LLM how to behave. It never changes, so it’s the same for every request.
User prompt: this changes with every request. It carries the actual question and the retrieved context.

We split them because the instructions are fixed and the user prompt is not. Keeping them apart makes the fixed part easy to reuse and the changing part easy to build fresh each time.

Instructions

The instructions tell the LLM its role and how to answer:

INSTRUCTIONS = """
Your task is to answer questions from the course participants
based on the provided context.

Use the context to find relevant information and provide accurate
answers. If the answer is not found in the context,
respond with "I don't know."
"""

This is what grounds the answer in our data and reduces hallucinations.

The user prompt template

The user prompt template has placeholders for the question and the context:

USER_PROMPT_TEMPLATE = """
Question:
{question}

Context:
{context}
"""

Building the context

The context is a formatted string with all the search results:

def build_context(search_results):
    lines = []

    for doc in search_results:
        lines.append(doc["section"])
        lines.append("Q: " + doc["question"])
        lines.append("A: " + doc["answer"])
        lines.append("")

    return "\n".join(lines).strip()

Each document becomes a block with the section, question, and answer. This format makes it easy for the LLM to read. We turned a list of dictionaries into one string. It’s a small preprocessing step before we send the data to the LLM.