Python Data Engineer

Key Responsibilities:

Data Pipeline Development: Design and implement data pipelines to extract, transform, and load (ETL) data from various sources into Azure-based data storage solutions.
Data Warehousing: Develop and maintain data warehouses and data marts, ensuring data integrity, performance, and accessibility.
Azure Cloud Expertise: Utilize Azure cloud services (such as Azure Data Factory, Azure Databricks, Azure SQL Data Warehouse, and Azure Synapse Analytics) to build scalable and reliable data solutions.
Python Development: Write clean, efficient, and maintainable Python code to automate data processes, perform data transformations, and support data analysis.
Data Quality and Governance: Implement data quality checks and data governance best practices to ensure data accuracy and compliance with regulations.
Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Performance Optimization: Identify and address performance bottlenecks in data pipelines and optimize data processing workflows.
Monitoring and Troubleshooting: Implement monitoring and alerting solutions to proactively identify and resolve data-related issues.
Documentation: Maintain comprehensive documentation for data pipelines, data models, and ETL processes.

Qualifications:

Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
7+ years of professional data engineering experience.
Strong proficiency in Python programming for data manipulation and automation.
Expertise in Azure Cloud services and technologies, including Azure Data Factory, Azure Databricks, and Azure SQL Data Warehouse.
Experience with data modeling and SQL databases.
Knowledge of data warehousing concepts and best practices.
Familiarity with data integration and ETL tools.
Excellent problem-solving skills and attention to detail.
Ability to work independently and as part of a collaborative team.

Bonus Skills:

Knowledge of big data technologies like Hadoop, Spark, or Kafka.
Experience with containerization technologies (e.g., Docker, Kubernetes).
Certification in Azure Data Engineering or related areas.