Lead Data Engineer

Job Description:

• Lead Data Engineer will play a critical role in ensuring the smooth and efficient operations of data management and analytics processes within the organization, overseeing data operations, implementing frameworks & standards, and providing continuous improvement in data-related workflows.

Responsibilities:

• Oversee the day-to-day operations including monitoring, data observability, data ingestion, regular checks for issues, anomalies, and bottlenecks & issue resolution.
• Support the team to take the capabilities of the platforms and put them into an operational practice.
• Collaborate with cross-functional teams, including data engineers, data scientists, business analysts, and IT teams, to understand data requirements and ensure data operations meet business needs.
• Collaborate with the data engineering team to optimize data workflows and ensure efficient data operations.
• Implement monitoring systems and processes to track the performance and observability of data.
• Identify bottlenecks, and proactively address performance issues to ensure high availability and responsiveness.
• Lead data incident management processes, including data outages, breaches, or data-related disruptions.
• Evaluate, select, and implement data management tools and infrastructure that support efficient data operations.
• Stay up-to-date with emerging cloud technologies and best practices in data management and operations to enhance data capabilities
• Identify opportunities for innovation and promote initiatives to enhance the data platform's capabilities and efficiency..
• Create and maintain comprehensive documentation, including data operation processes, standard operating procedures, and best practices.
• Design, implement, and sustain a DataOps framework, facilitate cross-team collaboration, optimize processes & tools, leverage testing automation.
• Mentor, monitor, and grow, including performance management of individual contributors.

Qualifications:

• 15+ years of experience in Data and Analytics in various roles where you have demonstrated the ability to collect, organize, analyze, and disseminate information
• 3+ years of experience in leading Data & Analytics Teams; experience in handling and leading teams, providing a collaborative and inclusive work environment
• Extensive experience in data operations, data management, or related roles
• Experience with data monitoring and performance optimization techniques
• Good experience in data operations, cloud infrastructure management, or data engineering roles, especially in a cloud-based environment
• In-depth knowledge of cloud platforms, AWS and related services for data storage, data processing, and infrastructure provisioning

Tech Stack:

• AWS, Python, Snowflake