Job Summary:
We're seeking a seasoned Data Consultant to design, build, and maintain efficient ETL processes from various data sources, ensuring the highest quality of data for our analytics. The role involves close collaboration with data consumers to capture requirements, develop strategic data load strategies, and ensure seamless data delivery through continuous improvement of our data pipeline and platform. The ideal candidate will be instrumental in pursuing data integrity, solving data-related issues, and enhancing our data handling capacities to meet evolving business needs.
Key Responsibilities:
- Develop and optimize ETL processes, ensuring reliable data extraction, transformation, and loading from multiple data sources using SQL and custom pipelines.
- Collaborate with stakeholders to understand requirements and create functional specifications for new data pipelines.
- Ensure data accuracy and integrity through thorough validation processes and troubleshooting of data discrepancies.
- Drive improvements in the data pipeline and platform, coordinating with Data Engineers to prioritize, develop, and test enhancements.
- Facilitate communication with business data analysts, providing technical and functional support to enable effective data access and analysis.
Must-Have Skills:
- Proficiency in SQL and advanced data warehousing techniques.
- Experience with big data sets, MPP database optimization, and Hive queries.
- Practical knowledge of AWS services related to data handling (Redshift, Athena).
Industry Experience:
- Prior experience in data engineering, particularly in the development and maintenance of ETL processes to support analytics in a large-scale data environment. Preferably with background or familiarity in handling clickstream data.