Data Integration Engineer/Analyst

Job Description:

• Seeking a skilled and motivated Data Integration Engineers to be part of existing dynamic team. The primary responsibility of this role is to lead the end-end process of acquiring, cleansing, transforming, and storing data from external vendors to AWS.
• The successful candidates will leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.

Responsibilities:

• Collaborate with cross-functional teams to understand data requirements and establish seamless connections with external vendors
• Develop and implement robust data acquisition strategies, ensuring data quality, integrity, and security.
• Lead the design and implementation of data cleansing and transformation process to align vendor data with internal standards and structures
• Utilize internal tools and SDKs for efficient data integration and processing, focusing on scalability and performance optimization
• Implement and manage agents for monitoring, logging, and automation within AWS environments
• Work with Amazon SNS for efficient notification and communication workflows
• Work with other AWS services, including S3, Athena, and Glue, to optimize data storage, querying, and processing.
• Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management
• Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.

Skills and Qualifications:

•  Bachelor’s degree in Computer Science, Information Technology, or related filed.
•  In-depth knowledge and hands-on experience with ASW services, including SNS, S3, Athena, Glue, and Lambda
• Expertise in data cleansing, transformation, and enrichment techniques
• Experience in deploying and managing Apache Kafka
• Proficient in programing language Java
• Excellent communication and collaboration skills
• Strong problem-solving skills and meticulous attention to detail.

Preferred Qualifications:

• AWS Certification.
• Advance expertise in Athena queries and optimization.
• Knowledge of data governance and compliance standards.