Data Engineer

Job Description:

Data Engineer (3+ years experience)

We are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with expertise in Azure Databricks, SQL, PySpark, and real-time data streaming using Kafka or other eventing solutions.

Responsibilities:

• Design, develop, and maintain data pipelines for ingesting, processing, and transforming large-scale data sets.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement efficient solutions.
• Optimize data pipelines for performance, scalability, and reliability.
• Implement real-time data streaming solutions using Kafka or similar technologies.
• Troubleshoot and resolve data pipeline issues.
• Ensure data quality and consistency across the platform.

Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 3+ years of experience in data engineering.
• Proficiency in Azure Databricks, SQL, and PySpark.
• Strong understanding of data modeling, ETL processes, and data warehousing.
• Experience with real-time data streaming using Kafka or other eventing solutions.
• Familiarity with cloud platforms (Azure, AWS, or GCP).
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.