Course Wiki

Evidently

The drift-and-quality evaluation tool MLOps Zoomcamp wires into both streaming and batch monitoring.

Evidently computes the monitoring signals: comparing the current batch of production data against a reference dataset and producing data drift metrics, target drift reports, and data quality checks. The module generates these as JSON metrics for machines and as dashboards for humans.

In the web-service pipeline, Evidently metrics are exposed on an endpoint for Prometheus to scrape every 15 seconds. In the batch pipeline, Evidently reports run over predictions stored in MongoDB. Either way, the tool turns raw prediction logs into named, thresholdable signals — the raw material for alerting.

Taught in

• LLM Zoomcamp

  • Module 5: Monitoring
    • Built-in Judge
    • Next Steps

• MLOps Zoomcamp

  • Module 5: Model Monitoring
    • Evidently metrics calculation
    • Evidently Monitoring Dashboard
    • Debugging with test suites and reports
  • Module 07: Course Project
    • Course Project ## Related concepts

• Model Monitoring

• Prometheus and Grafana

• Model Deployment

• Experiment Tracking