Job Title: Senior Data Pipeline Engineer (Python/Airflow)
Location: Guadalajara, Mexico
Duration: Long Term
Responsibilities
* Design, build, and maintain Airflow DAGs using TaskFlow, dynamic DAGs,
* deferrable operators, providers, and the Secrets backend; manage cross‑DAG dependencies and SLAs.
* Develop Python ETL/ELT code to ingest from APIs, object storage, message buses, and databases; package code as reusable libraries.
* Operate Airflow on managed or self‑hosted platforms (e.g., Azure, Kubernetes deployments); implement blue/green or canary DAG releases.
* Implement data quality and testing with unit tests for operators/hooks, and DAG validation in CI.
* Build event‑driven pipelines for near‑real‑time processing; manage schemas and compatibility.
* Model and manage data stores across SQL and blob storage; design partitioning, clustering, and retention.
* Observability & lineage: instrument metrics/logs, set SLAs/alerts, drive post‑incident reviews and reliability improvements.
* Security & governance: apply least‑privilege IAM, secrets management, PII
* handling, and data contracts; enforce RBAC in Airflow and warehouses.
* CI/CD & IaC: build pipelines to lint/test/deploy DAGs and Python packages;
* provision infra with Terraform/Helm; containerize with Docker.
* Cost & performance: tune task parallelism, autoscaling, storage formats, and compute footprints to optimize cost/perf.
* Collaboration: work closely with Android/backend teams to define interfaces and data contracts; document decisions and operational runbooks.
Skills and Qualifications
* 8+ years in data engineering or backend engineering with strong Python expertise.
* 2+ years Airflow 2.x expertise (operators, hooks, sensors, TaskFlow, scheduler
* tuning).
* Proven experience designing reliable ETL/ELT at scale (batch and streaming) with
* robust testing and monitoring.
* Strong SQL and data modeling skills; hands‑on with one or more data warehouses
* (BigQuery, Redshift, Snowflake) and relational systems (PostgreSQL/MySQL).
* Familiarity with security best practices (RBAC, OAuth2/OIDC for service
* integrations), API gateways, and secrets management (Vault/AWS Secrets Manager/GCP Secret Manager).
* Comfortable operating in production: monitoring, troubleshooting, and performance tuning.
* Excellent written and verbal communication; clear trade‑off communication and
* autonomous execution with well‑documented decisions. approach to problem solving.