Azure Data Engineer

Job Description

Azure Data Engineer:

At least 5-10 years of experience in development of data solutions using Azure cloud platform
Good understanding Massively parallel processing (MPP) systems, experience building Datawarehouse/DataMart on Azure Synapse SQL pools (SQL DW)
Strong SQL skills and experience writing complex yet efficient SPROCs/Functions/Views using T-SQL
Experience building ADF pipelines for ingestion from Flat files, RDBMS systems, Cloud storage, SFTP etc. Building complex orchestrations using ADF
Solid understand of spark architecture and experience with performance tuning big data workloads in spark
Building complex data transformations on both structure and semi-structured data (XML/JSON) using Pyspark & SQL, refactoring tradition ML model to run on spark framework 
Familiarity with Azure Databricks environment and deploying spark code in databricks cluster 
Good understanding of No SQL and its use case, Modelling No SQL schemas & containers, building integration to read/write to cosmos 
Familiarity with Cognitive Search/Elastic search and its use cases & building integrations to load data to Search services 
Experiences with other big-data solutions like Kafka, Hive, Hadoop etc is preferred 
Good understanding of Azure cloud ecosystem; Azure data certification of DP-200/201/203 will be an advantage 
Proficient with Visual Studio 19+, IntelliJ/eclipse and source control using GIT 
Experience using any one programming language like Python/C#
Good understanding of Agile, DevOps and CI-CD automated deployment (e.g. Azure DevOps, Jenkins)