AWS Databricks Architect

Responsibilities:
•    Leading the architecture and implementation of Databricks-based ETL frameworks using Apache NiFi for large-scale enterprise systems.
•    Designing and developing high-throughput data pipelines using Databricks and streaming technologies.
•    Implementing and enforcing architectural standards and frameworks to ensure a flexible and scalable data environment.
•    Collaborating with cross-functional teams to gather requirements, analyze data, and design effective solutions.
•    Hands-on development of Python and Java-based scripts and applications to support data processing and transformation.
•    Playing a key role in DevOps activities, including deployment of Spark jobs and infrastructure setup.
•    Providing mentorship and technical guidance to junior team members.
•    Staying updated with the latest industry trends and technologies in data engineering and analytics.
 
Qualifications:
•    Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
•    Overall experience of 8 – 12 years. 5+ years of hands-on IT experience with a strong focus on ETL and Python technologies.
•    Proven expertise in designing and implementing data solutions using Databricks, AWS.
•    Extensive experience with Apache NiFi and streaming development.
•    Architectural understanding and experience in Databricks running on AWS deployment.
•    Solid understanding of Python and Java programming for data manipulation and transformation.
•    Proven ability to troubleshoot and optimize Databricks queries for analytics and business intelligence use cases.
•    Basic knowledge of DevOps practices for managing Databricks job deployments.
•    Databricks Certified Certification in Advanced Data Engineering is a plus.
•    Strong problem-solving skills and the ability to work effectively in a collaborative team environment.
•    Excellent communication and interpersonal skills.