we areseeking a talented data engineer to join our data engineering & analytics team. In this role, you will collaborate with our clients to address their most complex needs, working alongside a team of highly skilled engineers and architects. You will also be involved in strategic enterprise data projects and large-scale implementations.
responsibilities
- support projects leveraging an agile delivery model through requirements gathering and documentation, data ingestion pipeline development, and technical testing.
- contribute to project delivery in the following ways:
- add configurations for data enrichments such as lookups, type conversions, calculations, and concatenations
- add data quality rules
- connect to new data sources
- map multiple sources into a single output
- interpret error messages to troubleshoot data pipelines
- create outputs that facilitate reporting, dashboarding, and semantic modeling for tools such as tableau and power bi
- test data pipelines to ensure accuracy
- work closely with our clients as part of the development team to gather technical requirements and develop solutions
- coordinate activities with data source application owners to understand the underlying data, ensure integration and data integrity
- build relationships with clients through written and verbal communication
qualifications
- 2+ years of hands-on professional development experience, consulting experience preferred.
- 1+ years of experience working with databricks
- experience writing complex sql queries related to data manipulation strategies
- experience building or working with metadata-driven etl / elt patterns and concepts.
- experience with spark, databricks, kafka, kinesis, hadoop, and/or lambda.
- exposure using etl tools (azure data factory, aws glue, informatica, talend, ibm datastage, etc.), database platforms (snowflake, redshift, azure synapse, or oracle), and data visualization tools (tableau, power bi, looker).
- experience in technical testing, issue identification and resolution
- strong verbal and written communication skills
