Key Responsibilities:
We are seeking a highly skilled and experienced onsite Senior Data Engineer to lead the design and implementation of a greenfield Order Management System (OMS). This role will be critical in shaping our data strategy, impacting our data lake and warehouse. The ideal candidate will have extensive experience building scalable data solutions, strong analytical skills, and a passion for leveraging data to drive business decisions.
Design and Implementation:
Lead the design and implementation of the OMS data architecture.
Develop and maintain data pipelines to ensure seamless data flow from the OMS to the data lake and data warehouse.
Ensure data integrity, consistency, and availability across all data systems.
Data Integration:
Integrate data from various sources, including transactional databases, third-party APIs, and external data sources, into the data lake.
Implement ETL processes to transform and load data into the data warehouse for analytics and reporting.
Collaboration:
Work closely with cross-functional teams including Data Science, Business Intelligence, and Product Management to understand data requirements and deliver solutions.
Collaborate with data engineers to ensure data engineering best practices are integrated into the development process.
Performance Optimization:
Optimize data storage and retrieval to improve performance and scalability.
Monitor and troubleshoot data pipelines to ensure high reliability and efficiency.
Data Governance:
Implement and enforce data governance policies to ensure data security, privacy, and compliance.
Develop documentation and standards for data processes and procedures.
Education:
Bachelor's or Master’s degree in Computer Science, Information Systems, or a related field.
Experience:
Minimum of 7-8 years of experience in data engineering, with a focus on data architecture and pipeline development.
Proven experience with cloud platforms (GCP) and big data technologies (e.g., Airflow, Spark, DBT,Databricks. BigQuery, GCP Servcies).
Hands-on experience with ETL tools and processes.
Strong SQL skills and experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
Skills:
Proficiency in programming languages such as Python
Experience with data modeling, schema design, and data warehousing concepts.
Familiarity with data lake architectures and best practices.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration skills.
Bachelor's degree in Computer Science