8+ years' experience working in a similar Senior Software Engineer role within a data platform/data engineering team
Extensive experience with Big Data and distributed systems
Experience with stream processing using Spark
Ability to design large-scale distributed systems
Prior experience in building data platforms using Big Data stack (Kafka, Hadoop, Spark, Flink, Hive ..) on the public cloud
Excellent programming skills in Java
Understanding of distributed systems concepts and principles (consistency and availability, liveness and safety, durability, reliability, fault-tolerance, consensus algorithms)
Deep understanding of Algorithms, Data Structures, and Performance Optimization Techniques
Eager to learn new things and passionate about technology!
Comfortable working with Kubernetes, AWS, Docker, and Terraform
What you would do:
Hands-on coding and implementation of high-performance Spark applications, leveraging your deep understanding of distributed computing and data processing
Designing and optimizing Spark applications
Design cloud-native data platform and analytics SaaS services
Own architecture and provide technical leadership to multiple teams
Hands-on coding>60% of the time
Design and build large-scale real-time stream processing systems
Design and develop micro-services and analytics SaaS solutions
Own scalability, availability, and data security
Own, tackle & resolve code defects
Mentor junior developers in standards and best practices!
What you would need to succeed:
Extensive experience working on a public cloud-like AWS
Prior experience and passion for building a large-scale multi-tenant cloud-native data platform
Emphasize team wins over individual success
Strong technical interpersonal skills
Excellent software development skills in one or more of the following languages: Java/Scala
Extensive experience with Big Data and distributed systems
Expertise in Spark or Flink, Kafka, and Hadoop ecosystem
Ability to design large-scale distributed systems
Have developed in more than one language and ready to pivot to any language/framework
Understand microservices architecture patterns pattern like Service Discovery/API Gateway/Domain Driven Design etc
Experience with containers technology
Ability to work in an agile fast-paced environment
BS or MS degree (Computer Science or Math)
Bonus
AWS (EMR, S3, Glue, Kinesis)
Debezium
ELK
Experience in building SaaS/PaaS on AWS/GCP/Azure
AI/ML
Bachelor's degree in Computer Science