Azure Data Engineer

Job Description

This is a remote position.

Job Description:
 

We are seeking a talented and experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in data engineering with a focus on Microsoft Azure services. The candidate should have a minimum of 5-10 years of hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, Synapse Pipelines, Synapse Spark, and Synapse SQL Pool.
 

Responsibilities:
 

• Data Ingestion and Integration:
   
• Design, develop, and implement data pipelines using Azure Data Factory to ingest, transform, and integrate data from various sources.
   
• Implement data integration solutions that meet business requirements and ensure data quality and reliability.
   

Azure Synapse Analytics:
 

• Work with Azure Synapse Analytics to design and implement data warehousing solutions.
   
• Develop and optimize Synapse SQL Pool (formerly known as SQL Data Warehouse) for efficient querying and reporting.
   

Synapse Pipelines:
 

• Create and manage Synapse Pipelines to orchestrate end-to-end data workflows.
   
• Monitor and optimize pipeline performance to ensure timely and reliable data processing.
   

Synapse Spark:
 

• Leverage Azure Synapse Spark for big data processing and analytics.
   
• Develop and optimize Spark jobs to handle large-scale data processing requirements.
   

Data Modeling:
 

• Design and implement data models to support analytical and reporting requirements.
   
• Ensure data models are scalable, performant, and aligned with business needs.
   

Collaboration:
 

• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
   

Documentation:
 

• Create and maintain comprehensive documentation for data engineering solutions, including design documents, process flow diagrams, and operational guidelines.
   

Performance Tuning:
 

• Identify and address performance bottlenecks in data pipelines, SQL queries, and Spark jobs.
   

Qualifications:
 

• Bachelor's degree in Computer Science, Information Technology, or a related field.
   
• 3-5 years of hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure  Databricks, Synapse Pipelines, Synapse Spark, and Synapse SQL Pool.
   
• Proficient in SQL and T-SQL.
   

·  Databricks Kafka