Description


" Architect/Design the Apache Airflow implementation.

" Provide SME in Airflow administration

" Single point of contact and liaison

" Build, maintain, monitor, and orchestrate workflows or data pipelines

" Provide environment/integration support to data platform/ tools

" Provide inhouse KT and training to other resources.

" Bring in best practices with respect to Airflow implementation.

" Guide team in Airflow implementation and orchestration of data pipelines. 3.Experience 8to12Yrs 4.Required Skills ,Kafka,Apache Airflow,Azure Synapse 5.Nice to have skills ,Python,Azure Kubernetes Service 6.Technology -Not Applicable 7.Shift Day 8.Roles & Responsibilities

" 3 to 5 years of industry experience as an Apache Airflow developer/architect

" Experience with data pipeline and workflow management tools: Airflow

" Experience with developing guidelines for Airflow clusters and DAGs

" Experience with Performance tuning of the DAG and task implementation

" Develop DAG - data pipeline to on-board and change management of datasets

" Experience installing Apache Airflow, configuring, and monitoring Airflow cluster

" Expertise in Python and its frameworks

" Understanding of airflow rest services and integration of airflow of platform eco-system

" Working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

" Experience building and optimizing - big data- data pipelines, architectures, and data sets.

" Experience of working with Azure Kubernetes/ Databricks/ Kafka/ Jenkins

" Working experience with CI-CD tools like Jenkins and Github code repositories

" Strong analytic skills related to working with unstructured datasets.

" Build processes supporting data transformation, data structures, metadata, dependency and workload management.

" Proficient in at least one and experience with modern programming languages (Java, C#, Python, JavaScript/TypeScript) and open-source technologies

" Possess professional software engineering practices and best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations

" Fast learner and a team player

" Working knowledge in Azure is required.

" Apache Airflow Fundamentals

" Agile/Scrum principles

" Excellent troubleshooting and debugging skills

" Ability to work independently as well as with multi-disciplinary teams
 

Key Skills
Education

ANY GRADUATE

Back To Jobs