Course Wiki

Terraform

Infrastructure as code for cloud environments: declarative resource definitions that can be planned, applied, and reproduced.

Module 1 pairs Docker with Terraform: after containerizing the runtime components, the cloud infrastructure itself — GCP storage buckets and BigQuery datasets for the course’s pipeline — is declared in Terraform configuration files instead of clicked into existence in a console.

The workflow is plan, apply, destroy. Plan shows what changes Terraform will make; apply creates the resources; destroy tears the environment down. For data engineers the payoff is reproducible environments: the same configuration recreates the same infrastructure for the next project or cohort.

Taught in

• MLOps Zoomcamp

  • Module 1: Introduction
    • Homework
  • Module 6: Best Practices
    • Homework
  • Module 07: Course Project
    • Course Project

• Data Engineering Zoomcamp

  • Module 1: Containerization and Infrastructure as Code
    • Introduction to GCP (Google Cloud Platform)
    • Introduction Terraform: Concepts and Overview, a primer
    • Terraform Basics: Simple one file Terraform Deployment ## Related concepts

• Docker

• BigQuery

• Kestra

• CI/CD

• Workflow Orchestration