Job description
must have
* python programming (advance)
* various data matching algorithms (string similarity based, distance-based, phonetics algorithm)
* basic graph db / vector db knowledge
* ml-based solution approach to define/ refine data matching criteria
* azure platform ( databricks / fabric / openai etc.)
good to have
* api programming (fastapi or flask)
* we have a need for two azure data engineers for a master data project. One person should be a senior lead engineer almost at architect level and second could be mid to senior engineer.
* the project requires data acquisition into azure, creating data pipelines, normalize json data into relational format.
* model and design table structures
* analyze data and identify relationships,
* match the data with another dataset to find patterns and relationship
* create combined datasets by using various matching and merging techniques and logics
* build apis to search the data
* build lightweight ui to enable search functionality for users
* main idea is we need self starter who can work independently and figure out the solution on their own, work with client team, discuss and modify accordingly.