Big Data Engineer

We are looking for a talented Big Data Engineer with strong expertise in ETL processes, AWS services, and Python programming to join our dynamic team. The ideal candidate will have experience working with large-scale data processing systems, building data pipelines, and optimizing data flows to help drive the success of our data-driven projects.

Key Responsibilities:

• Design, develop, and maintain robust ETL processes for extracting, transforming, and loading large datasets from multiple data sources.
• Build, optimize, and scale data pipelines and infrastructure to handle vast amounts of data efficiently.
• Work extensively with AWS services such as S3, EMR, Lambda, Glue, Redshift, and Kinesis to develop cloud-based big data solutions.
• Collaborate with data scientists, analysts, and other engineers to ensure high-quality data availability for analysis and business decision-making.
• Implement best practices for data management, data security, and compliance within the AWS cloud infrastructure.
• Monitor and maintain the performance and reliability of big data systems, identifying and resolving issues as needed.
• Write clean, maintainable, and efficient Python code to support data workflows and automation tasks.
• Stay up to date with emerging big data technologies and trends to continuously improve our data architecture and processes.