Big Data Engineer

Description: 

As a Senior Software Engineer in Data Platform & Engineering Services team, youll hold a valued role within a rapidly growing team inside one of the worlds most successful organizations, working closely with experienced and passionate engineers to solve problems customer problem. 
You will be partnering with the data engineering teams, so the ability to influence and provide operational guidance is key. 
Initially, the Developer focus will be contributing to the development of operational tools and practices that help maintain service availability across hosted and cloud-based infrastructure. 
You must understand the full stack and how systems are built as well as a grasp of operational best practices.

Role: 
As a member of the Unified Data Acquisition and Processing (UDAP) platform team, you will be responsible for building tools and systems that deploy and scale our applications and data in hybrid cloud and physical environments. 
We enable the platform that helps teams across multiple programs to build, test, deploy and host hundreds of data pipelines across several global datacenters along with enterprise logging, monitoring and vulnerability detection.

All about You/Experience: 
Experience in Data Warehouse related projects in product or service based organization
Experience solving for Scalability, Performance and Stability
Experience in a programming language in Java, Scala or Python
Experience working in SQL and relational databases
Operational experience in Big Data Stacks (Spark and Hadoop ecosystem)
Expert knowledge of Linux operating systems and environment and Scripting
A deep expertise in your field of Software Engineering
Expert at troubleshooting complex system and application stacks
Operational experience in Elastic Search (ELK stack) would be a plus
Operational experience troubleshooting network/server communication is a plus
Motivation, creativity, selfdirection, and desire to thrive on small project teams
Strong written and verbal English communication skills

What is the name of your group? How does that fit into the overall Client organization : 
Data Enablement team, UDAPP Unified Data Acquisition and Processing Platform

What program will this person be supporting? Will this person be a part of a Guild If so, which one and how will they be contributing: 
Yes, SDE

What is your teams main responsibility: 
Our team is responsible for building out data pipelines and processes for data movement from sources to their new warehouse.

How would you describe the culture of your team: 
What we look for is the person should be able to self research and can be autonomous in this position. 
Cannot depend on others to perform their work. 
Need the ability to research by oneself. 
They would work with product owners (data scientist) and enterprise architecture team, and data engineers. 

Would you please describe your management style: 
Hands on because was in a technical position previously, has thorough understanding. 
It's a small team but we all have great knowledge and experience. 
I'm open for all suggestions and changes to streamline/reorganize our approach to grow the business. 
We are building this platform and scaling it, but now we have lots of demand to use our platform. 
We are trying to take it to the next phase, and produce an additional platform to process cloud processing. 

What will a typical work day look like for this contractor: 
We do an alternating standup schedule, and their daily work will depend on their story.

What are your top 3 required technical skills : 
Java
Hadoop
Spark

What are a couple of desired/nice to have skills: 
Cloud Applications such as Azure and/or AWS 

What soft skills would you like to see in a candidate: 
Ability to research
Work autonomous
Willing to learn

What level of competency is required : 
Intermediate to Advanced