DataTalks.Club

Articles Slack Events Podcast Wiki Books Courses

Course Wiki

Module 6: Batch Processing — Data Engineering Zoomcamp

Lessons, materials, and key concepts for module 6 of Data Engineering Zoomcamp.

Related Pages

Data Engineering Zoomcamp

Data Engineering Zoomcamp > Module 6: Batch Processing

The lessons, homework, and materials of module 6 of Data Engineering Zoomcamp.

Lessons

LessonVideo
6.1.1 Introduction to Batch Processingvideo
6.1.2 Introduction to Sparkvideo
6.3.1 First Look at Spark/PySparkvideo
6.3.2 Spark Dataframesvideo
6.3.3 (Optional) Preparing Yellow and Green Taxi Datavideo
6.3.4 SQL with Sparkvideo
6.4.1 Anatomy of a Spark Clustervideo
6.4.2 GroupBy in Sparkvideo
6.4.3 Joins in Sparkvideo
6.5.1 Operations on Spark RDDsvideo
6.5.2 Spark RDD mapPartitionvideo
6.6.1 Connecting to Google Cloud Storagevideo
6.6.2 Creating a Local Spark Clustervideo
6.6.3 Setting up a Dataproc Clustervideo
6.6.4 Connecting Spark to Big Queryvideo

Key concepts

Homework and deadlines live in the course repository: data-engineering-zoomcamp.

Other modules in this course

DataTalks.Club. Hosted on GitHub Pages. Built with Rustkyll. We use cookies.