Data Engineer

Job Description:
We are looking for an experienced Data Engineer / Software Development Engineer in Test (SDET) with extensive experience in Python and cloud technologies, preferably AWS. The successful candidate will focus on automating ETL processes, managing data workflows, and collaborating with business stakeholders to deliver high-quality data applications.

Automate ETL processes using Python and AWS services.
Build and maintain data infrastructure with AWS tools (S3, Athena, EMR, Glue, Redshift).
Assist in migrating data from on-premises to the cloud.
Create SQL and Unix/Linux scripts for data processing.
Design and test ETL solutions using tools like Ab Initio and Informatica.
Analyze data sources and develop data quality reports.
Validate data movement and transformations between systems.
Collaborate with stakeholders to understand data needs.
Identify and resolve data issues.
Build automation frameworks for data testing.
Qualifications:
Bachelor’s degree in Computer Science, Data Science, or related field.
5+ years of experience in Data Engineering with a focus on Python and AWS.
Strong skills in SQL and Unix/Linux scripting.
Experience with ETL tools (e.g., Ab Initio, AWS Glue).
Familiarity with cloud data migration.
Background in DevOps/DataOps practices.
Knowledge of data science platforms (SageMaker, etc.) is a plus.
Skills and Technologies:
Languages: Python, SQL
Scripting: Unix/Linux Shell Scripting
Cloud: AWS (S3, Athena, EMR, Glue)
ETL Tools: Ab Initio, AWS Glue, Informatica
Version Control: Git, Jenkins
Data Warehousing: Understanding of data modeling concepts
Nice to Have:
Experience with test case management tools.
Familiarity with Agile methodologies.
Understanding of RESTful APIs