Data Engineer

Required Skills : Databricks/Spark/Scala, Python,Spark in Scala, Databricks,Databricks Delta Lake,Databricks SQL

We are seeking a highly skilled Sr. Data Engineer with 10 to 14 years of experience to join our team as a Business Associate - BC.
The ideal candidate will have extensive experience in Databricks, Databricks Delta Lake, and Databricks SQL, with additional skills in Spark in Scala and Python being a plus.
This role will involve designing and implementing data solutions that drive business success and innovation.

Roles & Responsibilities
Lead the design and implementation of data solutions using Databricks, ensuring high performance and scalability
Oversee the integration of Databricks Delta Lake to manage and optimize large datasets
Provide expertise in Databricks SQL to develop complex queries and data models
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
Ensure data solutions are aligned with business goals and objectives, driving efficiency and innovation
Develop and maintain documentation for data architectures, solutions, and processes
Monitor and optimize data pipelines to ensure data quality and reliability
Implement best practices for data governance and security, ensuring compliance with industry standards
Mentor and guide junior team members, fostering a culture of continuous learning and improvement
Conduct regular code reviews to ensure high-quality code and adherence to best practices
Stay updated with the latest trends and advancements in data technologies, incorporating them into the teams work
Provide technical support and troubleshooting for data-related issues, ensuring minimal downtime
Collaborate with stakeholders to identify opportunities for data-driven improvements and innovations

Qualifications
Possess extensive experience in Databricks, Databricks Delta Lake, and Databricks SQL
Have a strong understanding of data architecture and data engineering principles
Demonstrate proficiency in developing and optimizing data pipelines
Show expertise in data governance and security best practices
Exhibit strong problem-solving skills and the ability to troubleshoot complex data issues
Have experience in mentoring and guiding junior team members
Display excellent communication skills, with the ability to convey technical concepts to non-technical stakeholders
Be familiar with Spark in Scala and Python, with hands-on experience being a plus