Senior Data Engineer

Key Responsibilities:

Technical Leadership:
Lead and drive the delivery of scalable data and analytics solutions.
Lead a team of junior and senior data engineers.
Participate in solution design and architecture discussions.
Design, implement, and integrate new technologies.
Evolve data and analytics products.
Contribute to all aspects of data engineering from ingestion, and transformation, to consumption.
Build reusable frameworks, automated workflows, and libraries at scale to support analytics products.
Collaboration:
Work closely with internal partners, product owners, engineering leaders, data analysts, big data leads, and engineers.
Build positive relationships across product and engineering.
Solution Development:
Design and build reusable components, frameworks, and libraries at scale to support analytics products.
Implement complex automated workflows and routines using workflow scheduling tools.
Build continuous integration, test-driven development, and production deployment frameworks.
Clean, prepare, and optimize data at scale for ingestion and consumption.
Data Management:
Anticipate, identify, and solve issues concerning data management to improve data quality.
Drive the implementation of new data management projects and restructure the current data architecture.
Analyze and profile data for the purpose of designing scalable solutions.
Troubleshoot complex data issues and perform root cause analysis.
Mentorship:
Mentor and develop other data engineers in adopting best practices.
Drive collaborative reviews of design, code, test plans, and dataset implementation.

Required Skills and Experience:

Technical Expertise:
Minimum 8 years of experience developing scalable Big Data applications or solutions on distributed platforms.
Strong skills in SQL, Spark, Python, Airflow, and Scala.
Experience with cloud computing platforms (AWS, GCP, Azure).
Experience with Jenkins pipelines for CI/CD processes and building Docker images.
Working knowledge of data warehousing, data modeling, governance, and data architecture.
Experience with data platforms such as EMR, Airflow, and Databricks (Data Engineering Delta Lake components, and Lakehouse Medallion architecture).
Experience with data warehousing tools like SQL database, Presto, and Snowflake.
Experience with streaming, serverless, and microservices architecture.
Soft Skills:
Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.
Ability to influence and communicate effectively with team members and business stakeholders.
Ability to work independently and as part of a global team.
Self-motivated and able to work in a fast-paced environment.
Detail-oriented and committed to delivering high-quality work.
Strong skills in building positive relationships across product and engineering.