Spark Developer/ Data Engineer

Essential Skills/Basic Qualifications:
Skills required: Minimum hands on/proven experience 9+ years
his is a senior developer role. Developer(s) will contribute to the development and deployment of the client's state-of-the-art big data platform.
Strong Spark, Scala/Python, Kafka, and Azure Cloud
Skills & Experience Required:
• Scala/Python, Spark Structured Streaming, Spark Streaming, Kafka, micro-batching
• Azure Event Hubs, Databricks, Stream Analytics, Data Explorer, Monitors
• Reading, writing, and optimizing streaming data at scale – e.g. handling millions of connected devices sending billions of messages per day
• Reading from and writing to RDBMs and NoSQL DB
Strong understanding of the following:
• Event-Driven and Domain-Driven Architecture & Implementations
• Backend development of micro-service within the Cloud platform (in Azure)
Essential | Working Knowledge of:
• Designing experience in High Available and Reliable Solutions
• Observe, test, debug, and correct faults in a code base
• Develop and produce documentation, including technical specifications
• Hands-on experience in the unit test, integration tests, CI/CD process, and code coverage
Secondary Skills
• Kubernetes
• Full data management skills – data profiling, data analysis, data analytics, data transformation, data optimization, data governance, data anomaly handling
• Working knowledge in Yugabyte, Hadoop, HDFS, and relational databases
• Hands-on experience in Spark Streaming / Spark Structured Streaming