Data Engineer

Job responsibilities

• Design database and data pipeline/ETL using emerging technologies and tools
• Drive the team to develop operationally efficient analytic solutions
• Define standards, methodologies for the Data Warehousing environment
• Design and build highly scalable data pipelines using new generation tools and technologies like AWS, Snowflake, Spark, Kafka to induct data from various systems
• Translate complex business requirements into scalable technical solutions meeting data warehousing design standards
• Create scalable data pipelines and ETL applications that support business operations in advertising, content, and finance/accounting
• Assist with deciphering data migration issues and improving system performance
• Collaborate efficiently with product management, technical program management, operations, and other engineers

Minimum Requirements

• BS, MS, or Ph.D. in Computer Science or a relevant technical field
• Extensive experience building scalable data systems and data-driven products working with cross-functional teams
• 2+ years of related software/data engineering experience with a proficiency in Python
• Ability to create data pipelines and ETL applications with large data sets
• Proficiency in building REST APIs for back-end services
• Exposure to implementing, testing, debugging, and deploying data pipelines using any of the following tools: Prefect, Airflow, Glue, Kafka, Serverless(Lambda, Kinesis, SQS, SNS), Fivetran, or Stitch Data/Singer
• Experience with any of the Cloud data warehousing technologies: Redshift, BigQuery, Spark, Snowflake, Presto, Athena, S3
• Experience with SQL DB administration (PostgreSQL, MS SQL, etc.)

Preffered Skills

• Understanding of complex, distributed, microservice web architectures
• Experience with Python back-end, ETL to move from one database to another
• Solid understanding of analytics needs and proactive-ness to build generic solutions to improve the efficiency