Were looking for a highly skilled senior data engineer to join our team. This critical role involves managing massive volumes of high-velocity high-cardinality data generated by real-time processing systems primarily in the computer vision domain. The ideal candidate possesses deep expertise in timescaledb for efficient storage and querying proficiency with cloud-based object storage and familiarity with the specialized nvidia ecosystem including deepstream gpus and trident to ensure data integrity and flow from the edge to the analytics platform.
responsibilities:
data architecture & storage
* design build and optimize high-throughput data pipelines using modern tools to ingest streaming data from various sources into our core data platform.
* serve as the subject matter expert for timescaledb managing schema design performance tuning compression policies and data retention strategies for petabytes of time-series data.
* architect and manage the tiered storage strategy leveraging cloud-based storage solutions (e.g. S3 gcs) for cold storage and archival of raw and processed data.
* ensure data models are scalable and optimized for both real-time operational queries and large-scale analytical processing.
real-time & computer vision integration
* collaborate with ml and computer vision teams to integrate the data pipeline with nvidia deepstreamapplications managing metadata and telemetry extracted from video streams.
* develop solutions that utilize nvidia gpus effectively particularly concerning how derived data is ingested and processed immediately after the visual inferencing stage.
* familiarity with nvidia trident storage orchestration is desirable for managing persistent volumes in kubernetes environments hosting vision applications.
* implement data quality checks and validation processes to ensure the high integrity of timestamps and measurement data from the edge devices.
engineering excellence & collaboration
* apply expert-level proficiency in a major programming language (python or scala preferred) for etl/elt pipeline development and tooling.
* drive the adoption of best practices including infrastructure as code (iac) and comprehensive monitoring (e.g. Prometheus/grafana) for the data platform components.
* provide technical guidance and mentorship to junior team members fostering a culture of high performance and technical rigor.
qualifications :
* minimum of 5 years of professional experience in data engineering focusing on high-volume data platforms or distributed systems.
* expert proficiency with timescaledb (postgresql) including experience managing production instances hypertable partitioning and continuous aggregates.
* demonstrated experience designing and managing large-scale data lakes or warehouses utilizing cloud-based object storage (aws s3 azure blob storage or gcp cloud storage).
* deep experience with streaming platforms (e.g. Apache kafka flink) and real-time data ingestion patterns.
* proficiency in modern programming languages (e.g. Python scala or go) for data processing and pipeline orchestration.
preferred skills & domain knowledge
* familiarity with the nvidia computer vision stack including concepts related to deepstream nvidia gpus or edge-to-cloud data flow.
* experience or strong understanding of the requirements for storing and retrieving high-dimensional time-series data (e.g. sensor data telemetry and machine learning metadata).
* experience with containerization and orchestration (docker kubernetes) in the context of data processing jobs.
* familiarity with database security protocols and compliance requirements for sensitive data.
additional information :
perks you enjoy at kms mexico
* mexican law benefits
* 15 days of pto (in year zero from the first year onwards it is 3 days per year).
* 5 days leave for the death of immediate family members negotiable.
* major medical expenses insurance with coverage for immediate dependents (spouse and children).
* annual performance bonus (10% of annualized salary).
* annual salary adjustment.
* employee referral bonus.
* paid certifications / courses
* coursera license.
* 5% savings fund.
* 5% grocery vouchers.
remote work :
no
employment type :
full-time