Python Big Data Engineer

Must-Have Skills And Experience

Cloud Data Pipelines: 5+ years of designing and implementing data pipelines in a cloud environment.
AWS Cloud Solutions: 2+ years of experience migrating/developing data solutions specifically in AWS.
Databricks or Similar: 1+ year of experience building and implementing data pipelines using Databricks or a similar cloud database platform.
SQL Expertise: Expert-level SQL skills for writing complex, optimized queries across large datasets.
Python Programming: Hands-on experience with object-oriented programming in Python.
Real-Time Data Streaming: Professional experience building real-time data streams using Spark.
Data Lake Architecture: Knowledge or experience in architectural best practices for building data lakes.

Responsibilities

Design, develop, and maintain scalable and efficient data pipelines in the AWS cloud environment.
Migrate existing data solutions to AWS or build new ones from scratch.
Leverage Databricks (or similar) to construct and manage data pipelines.
Write and optimize complex SQL queries for data extraction, transformation, and loading.
Utilize Python for data processing and automation tasks.
Build and maintain real-time data streaming solutions using Spark.
Apply architectural best practices to the design and implementation of data lakes.
Collaborate with cross-functional teams to understand data requirements and deliver solutions.