Data Engineer

Job Responsibilities

·        Highly skilled and motivated Data Engineer

·        As a Data Engineer, you will be responsible for validating the data between two systems, ensuring accuracy and consistency.

·        Implement data quality checks and monitoring processes.

·        Design and Develop data entity validation scripts using python.

·        Validate and reconcile data between two systems, ensuring accuracy and consistency.

·        Collaborate with cross-functional teams to identify and resolve data discrepancies and issues.

·        Develop and implement data quality checks and monitoring processes to ensure data integrity.

·        Design and maintain data pipelines and ETL processes to extract, transform, and load data from various sources.

·        Optimize data storage and retrieval processes to improve performance and efficiency.

·        Perform data analysis and profiling to identify data quality issues and recommend solutions.

·        Develop and maintain documentation for data validation processes, data mappings, and data lineage.

·        Stay up to date with industry trends and best practices in data engineering and data validation.

·        Support migration of North American OPCOs, enabling data validation on the new platform.

·        Gather business requirements from key client stakeholders and translate these into technical tracking specifications based on standards and industry best practice.

Required Skills

·        Strong knowledge of IPAAS and data validation for migration of applications.

·        Demonstrable experience in delivering end to end data validation.

·        Confident understanding of web development and its languages and possess Python proficiency.

·        Proactive and highly organised with strong time management and planning skills. Able to change direction and work on multiple projects across a wide range of topics.

·        [Desired] Knowledge of Python/Google BigQuery architecture