Technical Lead Data Engineer

Preferred Qualifications:
- Knowledge of cloud platforms like GCP/AWS, building Microservices and scalable solutions, will be an advantage
- 1 + years of experience in designing and building solutions using Kafka streams or queues
- Experience with GitHub/Bitbucket and leveraging CI/CD pipelines
- Experience with NoSQL i.e., HBase, Couchbase, MongoDB is good to have
- Excellent technical and analytical aptitude
- Good communication skills
- Excellent Project management skills.

Job Summary:
- 10+ years of software development experience and leading teams of engineers and scrum teams
- 5+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL and PySpark)
- 5+ years of experience within Data Engineering/ Data Warehousing using Big Data technologies
- Knowledge of Financial reporting ecosystem will be a plus
- Expert on Distributed ecosystem
- Hands-on experience with programming using Core Java or Python/Scala

Roles & Responsibilities:
- Expert on Hadoop and Spark Architecture and its working principle
- Hands-on experience on writing and understanding complex SQL(Hive/PySpark-dataframes), optimizing joins while processing huge amount of data
- Experience in UNIX shell scripting
- Ability to design and develop optimized Data pipelines for batch and real time data processing
- Should have experience in analysis, design, development, testing, and implementation of system applications
- Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows.