Python+Pyspark

Job Description

We are seeking a skilled Python Developer with a strong background in PySpark to develop and optimize data processing applications. The ideal candidate will be responsible for building robust and scalable data processing solutions using Python and PySpark. 

Requirements: 
 

Develop, maintain, and optimize scalable data processing applications using Python and PySpark.  
 

Design and implement data solutions that meet performance and reliability requirements. 
 

Collaborate with data engineers, data scientists, and other stakeholders to gather requirements and deliver high-quality solutions.  
 

Write clean, efficient, and maintainable code following best practices and coding standards. 
 

 Perform data analysis and ensure data quality and integrity.  
 

Monitor and troubleshoot performance issues in the data processing pipelines. 
 

 Implement and maintain CI/CD pipelines for automated testing and deployment.  
 

Stay up-to-date with the latest industry trends and technologies in Python and PySpark 
 

Required Skills and Qualifications: 
 

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.  
 

Proven experience as a Python Developer with expertise in PySpark.  
 

Strong knowledge of Python and its libraries (e.g., Pandas, NumPy). 
 

 Experience with Apache Spark, including Spark SQL, DataFrames, and Spark Streaming. 
 

 Proficiency in SQL and experience with relational databases.  
 

Familiarity with big data tools and frameworks. 
 

 Experience with version control systems such as Git.  
 

Strong problem-solving skills and attention to detail.  
 

Excellent communication and teamwork skills