Empleo
Mis anuncios
Mis alertas
Conectarse
Encontrar un trabajo Tips empleo Fichas empresas
Buscar

Senior data pipeline engineer (python/airflow)

Global Connect Technologies
Publicada el 9 diciembre
Descripción

Role summaryown the design and operation of reliable, secure, and cost‑efficient data pipelines builtwith apache airflow (2.x) and python. You’ll deliver batch and streaming ingestion,transformations, and curated datasets that power connected and infotainmentexperiences. You’ll lead python-based etl/elt, dag orchestration, and data platformreliability, security, and observability across our cloud environments.we are an android app development team looking for someone to own and lead our clouddata engineering in close partnership with mobile, backend, and product teams. Thisownership would include taking abstract requirements, refining/defining them, creatingdevelopment tasks, and then implementing those tasks.responsibilities• design, build, and maintain airflow dags using taskflow, dynamic dags,deferrable operators, providers, and the secrets backend; manage cross‑dagdependencies and slas.• develop python etl/elt code to ingest from apis, object storage, message buses,and databases; package code as reusable libraries.• operate airflow on managed or self‑hosted platforms (e.g., azure, kubernetesdeployments); implement blue/green or canary dag releases.• implement data quality and testing with unit tests for operators/hooks, and dagvalidation in ci.• build event‑driven pipelines for near‑real‑time processing; manage schemas andcompatibility.• model and manage data stores across sql and blob storage; design partitioning,clustering, and retention.• observability & lineage: instrument metrics/logs, set slas/alerts, drivepost‑incident reviews and reliability improvements.• security & governance: apply least‑privilege iam, secrets management, piihandling, and data contracts; enforce rbac in airflow and warehouses.• ci/cd & iac: build pipelines to lint/test/deploy dags and python packages;provision infra with terraform/helm; containerize with docker.• cost & performance: tune task parallelism, autoscaling, storage formats, andcompute footprints to optimize cost/perf.• collaboration: work closely with android/backend teams to define interfaces anddata contracts; document decisions and operational runbooks.skills and qualifications• 8+ years in data engineering or backend engineering with strong python expertise.• 2+ years airflow 2.x expertise (operators, hooks, sensors, taskflow, schedulertuning).• proven experience designing reliable etl/elt at scale (batch and streaming) withrobust testing and monitoring.• strong sql and data modeling skills; hands‑on with one or more data warehouses(bigquery, redshift, snowflake) and relational systems (postgresql/mysql).• familiarity with security best practices (rbac, oauth2/oidc for serviceintegrations), api gateways, and secrets management (vault/aws secretsmanager/gcp secret manager).• comfortable operating in production: monitoring, troubleshooting, andperformance tuning.• excellent written and verbal communication; clear trade‑off communication andautonomous execution with well‑documented decisions.nice to have• proficient with ci/cd, git, code reviews, and infrastructure as code (terraform);containerization with docker and orchestration on kubernetes is a plus.• spark (pyspark) or beam for large‑scale processing.• automotive/iot telemetry domain exposure.• experience with kafka (or event hubs/pub/sub equivalents), schema registry, andcdc patterns.• dbt for transformations and testing; delta lake/medallion patterns; feature stores.logistics• contract role; collaborate across multi‑disciplinary teams in a fast‑moving,target‑oriented environment.• motivated team player with high attention to detail and a creative, pragmaticapproach to problem solving.

Aplicar
Crear una alerta
Alerta activada
Guardada
Guardar
Ofertas similares
Empleo México
Inicio > Empleo > Senior data pipeline engineer (python/airflow)

Jobijoba

  • Tips empleo
  • Opiniones Empresas

Ofertas de empleo

  • Ofertas de empleo por ocupaciones
  • Búsqueda de empleo por categorías
  • Empleos por empresas
  • Empleos para localidad

Contacto / Asociados

  • Contacto
  • Publique sus ofertas en Jobijoba

Menciones legales - Términos y condiciones de uso - Política de Privacidad - Gestionar mis cookies - Accesibilidad: No conforme

© 2025 Jobijoba - Todos los derechos reservados

Aplicar
Crear una alerta
Alerta activada
Guardada
Guardar