Technology Lead - Azure Databricks - Data Engineer

Technology Lead - Azure Databricks - Data Engineer

Raleigh,United States

Richardson,United States

Permanent

Skills

Azure

ADF

Hadoop

databricks

show less

Required Qualifications:

Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.

At least 4 years of experience in Information Technology.

At least 3 years of hands-on experience in using Azure Data Factory (ADF) to orchestrate data ingestion from various sources into Azure Synapse and Azure Data Lake Storage (ADLS). 

At least 2 years of hands-on experience with Azure, databricks and Hadoop distributed frameworks while handling large amount of big data using Spark and Hadoop Ecosystems.

At least 2 years of experience designing and implementing data pipelines using Azure Databricks for data cleaning, transformation, and loading into Azure Synapse Analytics

Hands on experience in end-to-end implementation of data warehouse and data marts

Experience in writing SQL queries to analysis Type-2 dimension data

Understanding of implementing Medallion Architecture using Azure Services

Preferred Qualifications:  

Experience in software engineering and data engineering roles, with a focus on Azure and Databricks.

Strong understanding of databases and big data software technologies, specifically Azure and Databricks.

Exposure to Unity Catalogue

Experience with Agile methodologies, particularly Scrum.

Experience in coordinating with offshore data engineering teams

Experience with scripting languages like Python, Java, etc.

Strong problem-solving skills related to data, data structures, and algorithms.