Data Engineer (Gen AI)

Key Responsibilities

Design, construct, install, test, and maintain data management systems
Develop and maintain scalable data pipelines for data analytics and processing
Implement ETL processes to move data from various sources to data warehouses
Collaborate with data scientists and analysts to understand data needs and deliver appropriate solutions
Assess and develop data architecture strategies for the organization's data ecosystem
Optimize and improve existing data pipelines and infrastructure
Perform data modeling and schema design for various application requirements
Ensure data quality and integrity by implementing best practices and data governance processes
Automate data processes and monitor system performance and reliability
Work closely with cross-functional teams to understand data requirements and provide technical solutions
Troubleshoot and resolve data-related issues and provide technical support as needed
Stay updated with the latest trends in data engineering and technologies
Contribute to the continuous improvement of data engineering best practices and standards

Required Qualifications

Strong experience in Python programming language - Must. 
Experience in REST APIs, Fast APIs, Graph APIs, SQL Alchemy. 
Good experience in Azure Services like DataLake, Azure SQL, Function App, Azure Cognitive Search. 
Good Understanding on Chunking, Embeddings, vectorization, indexing, Prompting, Hallucinations, RAG. 
Hands on experience in DevOps, create pull PRs and maintain code repositories. 
Strong communication skills and ability to collaborate effectively with team members.