Data Engineer

About the Role

building and optimizing data pipelines and data solutions utilizing Azure data services. This role requires a deep understanding of Azure cloud technologies, proficiency in data engineering, and experience with Azure Data Factory, Azure Databricks, Azure Data Lake Analytics, Azure Analysis Services, and Azure Synapse

Key Responsibilities:

Data Pipeline Development: Design, develop, and maintain data pipelines using Azure Data Factory, ensuring data is processed, transformed, and loaded efficiently

Data Transformation: Implement data transformation logic using Azure Databricks, enabling data enrichment, cleansing, and preparation for analytics and reporting

Data Lake Management: Manage and optimize data storage and access in Azure Data Lake Storage, ensuring data is organized and accessible for analytics

Data Analytics: Develop and maintain data models and cubes using Azure Analysis Services for efficient data querying and reporting

Big Data Processing: Utilize Azure Synapse for big data processing and analytics to support advanced data insights

Data Governance: Implement data governance and data quality measures to maintain data accuracy, consistency, and security

Qualifications:

Bachelor's or Master's degree in computer science, Information Technology, or a related field (or equivalent work experience)

7-10 years of hands-on project development in the related areas of Data Engineering

5-7 years of hands-on experience development experience in Data Services

3-4 years Data Factory, Databricks, Data Lake Analytics, Analysis Services, Synapse

Experience in Data pipeline processing troubleshooting

Required Skills:

Strong programming and scripting skills (e.g., Python, SQL, Scala)

Knowledge of data modeling, data warehousing, and ETL processes

Experience in big data processing and analytics

Data governance and data quality expertise

Relevant Azure certifications are advantageous

Familiarity with Azure DevOps and CI/CD pipelines for data engineering is a plus

Relevant experience in Azure Data Factory or Synapse

Relevant experience in Databricks or Spark