Overview
as part of the data engineer team, you will be responsible for design, development and operations of large-scale data systems operating at petabytes scale. You will be focusing on real-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. You will interact with the engineers, product managers, qa, bi developers and architects to provide scalable robust technical solutions.
responsibilities
* design, develop, implement and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low-latency, and fault-tolerance in every system built.
* provide and support the implementation and operations of the data pipelines and analytical solutions
* experience in rest api data service – data consumption
* experience in managing work teams
* advanced project management
mandatory skills
* 8 to 10 years in technology project implementation
* english conversational (advanced)
* demonstrates up-to-date expertise in data engineering, complex data pipeline development
* experience in agile models
* experience with python, java to write data pipelines and data processing layers
* experience in advanced pipelines with airflow
* experience with continuous integration, devops, github
* performance tuning experience of systems working with large data sets
* proven working expertise with big data technologies hadoop, hive, kafka, presto, spark
* highly proficient in sql
* experience with cloud technologies
* gcp – dataproc, bigquery, cloud functions
* experience with relational models, memory data stores desirable (sql server, oracle, cassandra, druid)
* knowledge in implementing advanced analytics models using ml/ai (desirable)
* knowledge of bi tools (power bi, tableau, looker, etc.) Desirable
* retail experience is a huge plus
benefits
* legal benefits and superior legal benefits
* training and learning paths
seniority level
* associate
employment type
* full-time
job function
* information technology
industries
* it system data services and data infrastructure and analytics
#j-18808-ljbffr