Senior Software Engineer (Spark) - Cloud Data Platform

Job Description

Required Skills:

8+ years' experience working in a similar Senior Software Engineer role within a data platform/data engineering team
Extensive experience with Big Data and distributed systems.
Experience with stream processing using Spark.
Ability to design large-scale distributed systems.
Prior experience in building data platforms using Big Data stack (Kafka, Hadoop, Spark, Flink, Hive ..) on the public cloud.
Excellent programming skills in Java.
Understanding of distributed systems concepts and principles (consistency and availability, liveness and safety, durability, reliability, fault-tolerance, consensus algorithms).
Deep understanding of Algorithms, Data Structures, and Performance Optimization Techniques.
Eager to learn new things and passionate about technology!.
Comfortable working with Kubernetes, AWS, Docker, and Terraform.

What You Would Do

Hands-on coding and implementation of high-performance Spark applications, leveraging your deep understanding of distributed computing and data processing.
Designing and optimizing Spark applications.
Design cloud-native data platform and analytics SaaS services
Own architecture and provide technical leadership to multiple teams
Hands-on coding>60% of the time
Design and build large-scale real-time stream processing systems
Design and develop micro-services and analytics SaaS solutions
Own scalability, availability, and data security
Own, tackle & resolve code defects
Mentor junior developers in standards and best practices!

What You Would Need To Succeed

Extensive experience working on a public cloud-like AWS
Prior experience and passion for building a large-scale multi-tenant cloud-native data platform
Emphasize team wins over individual success
Strong technical interpersonal skills
Excellent software development skills in one or more of the following languages: Java/Scala
Extensive experience with Big Data and distributed systems.
Expertise in Spark or Flink, Kafka, and Hadoop ecosystem
Ability to design large-scale distributed systems
Have developed in more than one language and ready to pivot to any language/framework
Understand microservices architecture patterns pattern like Service Discovery/API Gateway/Domain Driven Design etc
Experience with containers technology
Ability to work in an agile fast-paced environment
BS or MS degree (Computer Science or Math)