Spark Developer

Job Summary:
We are seeking a skilled and motivated Spark Developer to join our team. The ideal candidate will have experience in big data processing and analytics, with a strong background in Apache Spark. You will be responsible for designing, developing, and maintaining scalable and efficient data processing solutions.

Key Responsibilities:

• Develop and maintain large-scale data processing pipelines using Apache Spark.
• Optimize and tune Spark jobs for performance and scalability.
• Collaborate with data engineers, data scientists, and other stakeholders to gather requirements and translate them into technical solutions.
• Implement data integration solutions to extract, transform, and load data from various sources.
• Ensure data quality and integrity through rigorous testing and validation.
• Troubleshoot and resolve issues related to data processing and job execution.
• Stay up-to-date with the latest industry trends and best practices in big data technologies.

Required Skills and Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Proven experience as a Spark Developer or in a similar role.
• Strong programming skills in Scala, Java, or Python.
• Proficiency in using Apache Spark for data processing and analytics.
• Experience with big data technologies such as Hadoop, Hive, and Kafka.
• Familiarity with data storage solutions like HDFS, S3, or similar.
• Knowledge of SQL and experience with relational databases.
• Ability to work independently and as part of a team.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities