Data Analyst

Job Brief

We have an urgent requirement of a Data Analyst with 5 to 8 years of experience.

Key Responsibilities

Maintain and further develop the current ETL pipeline
Design, develop, and maintain ETL processes to extract data from source systems, transform it into the required format, and load it into the target databases or data warehouses.
Create and optimize data pipelines for real-time, batch, and near-real-time data processing.
Write complex SQL queries, scripts, and stored procedures for data manipulation and transformation.

Data Integration

Integrate data from multiple sources (e.g., APIs, flat files, databases, cloud platforms) into centralized storage.
Ensure that the data is properly transformed according to business logic and validation rules.

Data Quality & Validation

Implement data quality checks and validation routines to ensure the accuracy and reliability of the data.
Detects and resolves data discrepancies, inconsistencies, or errors.

Data Optimization & Performance

Monitor ETL jobs for performance and efficiency. Optimize ETL processes to reduce processing time and resource usage.
Troubleshoot and resolve performance issues related to ETL pipelines.

Technical Expertise

Our current pipeline is a combination of Python and SQL. Should have expertise in both.
Experience with relational databases like SQL Server, PostgreSQL
Experience with cloud platforms like AWS ( Athena, Kubernetes )

Preferred Skills

Design, implement, and manage CI/CD pipelines
Automate manual processes related to software build, testing, deployment, and configuration management.
Deploy and manage applications on cloud platforms like AWS
Build and maintain scalable, resilient cloud-based infrastructure using services like EC2, S3, Kubernetes, and Docker.
Implement container orchestration solutions with Kubernetes, Docker