Back to Portfolio
L’OréalData Engineering90% Reduced Incidents

Automated Data Quality Control Tower

Building a proactive validation system using Python and Google Sheets to catch data issues before stakeholders, restoring trust in the data ecosystem.

PythonGoogle SheetsBigqueryOutlook API

The Challenge

Data quality issues were often discovered "too late", usually by stakeholders during executive meetings.

  • The Symptom: Management lost trust in the dashboards.
  • The Root Cause: The team was reactive, fixing issues only after they were reported. There was no automated pre-morning check to verify completeness across the 20+ critical data points.

The Stack

  • Configuration: Google Sheets (The "Control Center" for defining rules).
  • Compute: Python (The Validation Engine).
  • Storage/Target: BigQuery (The Data Source).
  • Alerting: Outlook API (Notification System).

The Architecture

Image

The Solution

Proactive "Control Tower"

I built a lightweight, automated validation framework designed to run before the business day starts (6:00 AM).

  1. The Control Center: I used Google Sheets as the configuration layer. This allowed the team to add new validation rules (e.g., "Sales cannot be zero," "Inventory must match yesterday") without touching the code.
  2. The Logic: A Python script reads the rules, queries BigQuery, and detects anomalies.
  3. The Action: If a check fails, the script uses the Outlook API to immediately email the specific PIC (Person In Charge) with a "Fix It" ticket.

The Code (System Logic) → The script bridges the gap between flexible configuration (Sheets) and strict enforcement (Python).

The Result

  • Reliability: Reduced reported data incidents by 90%.
  • Culture: Shifted the team from "Firefighting" to "Prevention."
  • Trust: Restored stakeholder confidence, as errors were fixed before dashboards were opened.