Data Engineer_Pyspark

Required Skills : Unix Shell Scripting,Unix,PySpark,Spark Pyspark

Job summary
We are seeking a Sr. Business Analyst with 5 to 7 years of experience to join our team as a Business Associate.
The ideal candidate will have strong expertise in PySpark, SparkPyspark, Unix, and UnixShellScripting.
This role involves analyzing business processes, identifying areas for improvement, and implementing data-driven solutions to enhance operational efficiency.

Roles & Responsibilities
Analyze business processes to identify areas for improvement and optimization.
Develop and implement data-driven solutions using PySpark and SparkPyspark.
Collaborate with cross-functional teams to gather and document business requirements.
Provide insights and recommendations based on data analysis to support decision-making.
Create and maintain Unix and UnixShellScripting scripts to automate processes.
Ensure data integrity and accuracy through rigorous testing and validation.
Monitor and troubleshoot data pipelines to ensure smooth operation.
Prepare detailed reports and presentations to communicate findings and recommendations.
Conduct training sessions for team members on new tools and processes.
Stay updated with the latest industry trends and technologies to drive innovation.
Participate in project planning and management to ensure timely delivery of solutions.
Work closely with stakeholders to understand their needs and provide tailored solutions.
Maintain documentation of processes, solutions, and best practices for future reference.

Qualifications
Possess strong analytical skills with experience in PySpark and SparkPyspark.
Demonstrate proficiency in Unix and UnixShellScripting for process automation.
Exhibit excellent communication and collaboration skills to work effectively with cross-functional teams.
Show ability to manage multiple tasks and projects simultaneously with attention to detail.
Have a solid understanding of data analysis and visualization techniques.
Display a proactive approach to problem-solving and continuous improvement.
Hold a bachelors degree in Computer Science, Information Technology, or a related field.
Preferably have experience in the financial or technology industry.
Show commitment to staying updated with industry trends and best practices.
Demonstrate ability to work independently and as part of a team.
Exhibit strong organizational and time management