Our healthcare/pharmaceutical company is seeking a highly skilled Senior Data Engineer with expertise in software development, AWS cloud, Python programming language, Airflow/Kafka, and event-driven architecture to join our team. The successful candidate will work closely with AI scientists and data analysts to design, develop, and maintain data pipelines and systems that support clinical and operational data use cases.
Key Responsibilities:
- Design, develop and maintain scalable and efficient data pipelines and systems to support clinical and operational data use-cases
- Utilize Airflow/Kafka and event-driven architecture to build efficient data pipelines that handle large amounts of data in real-time
- Work collaboratively with AI scientists and data analysts to gather and analyze business requirements and ensure data solutions align with business needs
- Develop and maintain data warehouse and ETL processes, ensuring data quality and integrity
- Implement and maintain data security, data privacy, and compliance measures
- Perform data analysis, troubleshoot data issues, and provide technical support to end-users
- Collaborate with AI scientists and data analysts to integrate predictive models into the analytics platform
- Continuously monitor and optimize data pipelines and systems to ensure optimal performance and scalability
- Stay up to date with the latest technologies and trends in data engineering, AWS cloud, and healthcare/pharmaceutical industry.
Requirements:
- Bachelor's degree in computer science, Information Technology, or related field
- 5+ years of experience in software or data engineering
- Proven experience in designing and developing data pipelines and systems in a cloud environment, preferably AWS
- Strong proficiency in Python programming language and related frameworks/libraries
- Expertise in Airflow/Kafka and event-driven architecture
- Familiarity with database technologies such as SQL, NoSQL, and data warehousing concepts
- Experience with data security, privacy, and compliance measures
- Strong analytical and problem-solving skills
- Excellent communication and collaboration skills
- Knowledge of healthcare/pharmaceutical industry and related data standards and regulations is a plus.