Big Data Engineer with Scala Experience

Responsibilities:

• Develop, test, and deploy data processing applications using Scala and big data technologies.
• Collaborate with data scientists and analysts to design and implement data pipelines that support real-time and batch data processing.
• Ensure data quality and consistency across all data sources and pipelines.
• Optimize data architecture for performance, scalability, and reliability.
• Monitor and troubleshoot data processing issues to ensure uninterrupted data flow.
• Document processes, data structures, and pipeline architectures for ongoing maintenance and scalability.

Must-Have Skills:

• 3+ years of experience in Big Data engineering or related roles.
• Proficiency in Scala for data processing tasks.
• Hands-on experience with big data frameworks, such as Apache Spark, Hadoop, and Kafka.
• Strong understanding of data modeling and data warehousing concepts.
• Experience with ETL pipelines and data ingestion processes.
• Proficiency in SQL and database management systems.
• Familiarity with cloud platforms (e.g., AWS, Azure, or Google Cloud) for big data deployments.
• Experience with real-time data processing and stream processing solutions.
• Solid understanding of data security and data privacy practices.

Nice-to-Have Skills:

• Knowledge of functional programming principles in Scala.
• Experience with NoSQL databases like Cassandra, HBase, or MongoDB.
• Familiarity with containerization technologies such as Docker and Kubernetes.
• Knowledge of data governance frameworks and data cataloging tools.
• Certification in cloud platforms (AWS, Azure, or Google Cloud).
• Experience with data visualization tools (Tableau, Power BI) for data insights.