Senior Spark Engineer

Senior Spark Engineer

Remote Job |   2021-08-12 08:44:56

Apply Now

Share Job 

Job Code : spark

We have already won this project. No external competition. Immediate interview and start.

Ohio, Remote

Video interview

Must have excellent communications.

This is a senior-level role

Senior Spark Engineer to create the first POC around their planned use for Spark. 

Strong experience doing setup and configuration.  

Strong ability to setup the Spark cluster, establish best practices and work through lead level functions to get it set up correctly for function, integration and performance.  

So on the technical side, ideally:

• Spark cluster configuration and performance
• Presto configuration and performance
• pySpark and Spark based ML and Python notebooks
• Kubernetes
• Kafka/Pulsar familiarity helpful
• Devops automation Python, ansible, terraform etc.
• Oracle database performance + S3/Spark/Presto integration
• Some knowledge of Hadoop infrastructure
• Able to write shell script / python glue as needed
• Expertise with log aggregation, management and processing at scale