DATA ENGINEER

Overview

The Data Engineer plays a crucial role in the organization as they are responsible for designing, developing, and maintaining the company's data architecture and infrastructure. They work closely with data scientists and analysts to ensure data is properly collected, stored, and accessible for analysis and reporting.

Key Responsibilities
 

Design and implement data pipelines and ETL processes

Develop and maintain databases, data warehouses, and data lakes

Collaborate with data scientists to translate algorithms into production systems

Create and maintain optimal data pipeline architecture

Identify, design, and implement internal process improvements

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources

Work with stakeholders including the executive, analytics, and data engineering teams to assist with data-related technical issues

Design and implement data models and data architecture

Develop and optimize SQL queries

Monitor and maintain data quality, integrity, and security

Implement and maintain data governance and privacy measures

Keep up-to-date with the latest technologies and best practices in data engineering

Document and communicate technical solutions and best practices to the team

Required Qualifications

Bachelor's degree in Computer Science, Information Technology, or related field

Proven experience as a Data Engineer or similar role

Hands-on experience with cloud platforms such as AWS, Azure, or Google Cloud

Proficiency in ETL tools, data modeling, and database design

Strong programming skills in languages like Python, SQL, or Java

Experience with big data technologies such as Hadoop, Spark, or Kafka

Experience in data pipeline orchestation and automation

Experience in implementing and managing machine learning pipelines is desirable