Hadoop DevOps Engineer

Business Justification

SDP Devops and Deployment /Application Support Lead will be responsible for leading Devops and Application Support activities across all tenants within SDP

Position Summary

SDP Devops and Deployment /Application Support Lead will be responsible for leading Devops and Application Support activities across all tenants within SDP. He/ she will be responsible for building and enhancing the tools needed for proactive monitoring, troubleshooting, automating deployments and operations, and continuously developing tools to monitor and improve the availability of the data on the platform.  He/She will drive Devops automation and increase deployment efficiencies.

Resource will deploy code changes to the test and production environments using Ansible Tower and related tools in SDP. Will also provide support to the Release Management team during production release events. Additional duties include tracking/updating tickets using JIRA, working with application and platform support teams to triage/fix issues with the deployments, and creating/updating documentation to support team functions. Working release weekends is expected (~once a month).

Primary Skill

Agile tools (Bitbucket)

Secondary Skill

Hadoop

Tertiary Skill

Agile tools (Jenkins)

Required Skills

• Expert in Reliability Engineering, Incident management, Observability, monitoring, and Root Cause Analysis.  Use Tune alerting mechanisms and setup observability tools to proactively identify the issues and performance problems. The ideal candidate should be proficient in tenant application production operational processes in a Hadoop environment, and should have the ability to handle multiple tenants and provide oversight leadership across multiple self-service support teams.
• The ideal candidate will be a bridge between these L1,L2 and L3 level support across the largest instance in the bank (50 TB+ SDP - Strategic Data Platform) that has close to 100 tenant applications, and can communicate technically, demonstrate leadership in resolving problems for production successful operations
• Hands-on and strong understanding of Hadoop architecture is required. Experience with Hadoop ecosystem components - HDFS, YARN, MapReduce & cluster management tools like Ambari or Cloudera Manager, Pepper Data and related technologies.
• Excellent Shell, Python programming skills for automation requirement for repetitive dev-ops tasks. DevOps + Deployment + Production Ops. Proficiency in scripting, Linux system administration, networking, and troubleshooting skills.

· 5+ years of Ansible automation and DEVOPS Engineering experience

· Strong Knowledge of GIT (and repo) source control and Artifactory

· Experience with Continuous Integration (CI) and Continuous Deployment (CD) automation and CI/CD toolchain

· Strong Knowledge of XLR

· 2+ years experience using CI/CD pipeline to deliver infrastructure as code

·  Additional duties include onboarding new applications onto the CD pipeline using the existing toolset ( Celestial, Tower & XLR)

· Experience with documenting/designing/implementing Linux infrastructure components and Linux based solutions

· Strong knowledge of Unix/Linux operating systems

Desired Skills

· Experience with product release cycle and release management

· Previous experience in the financial services industry

· Understanding of industry trends and relevant application technologies

· Experience in implementing Devops tools and program solutions 

· Excellent communication skills (written and verbal) and enterprise experience

· Exceptional problem solving and time management skills

· Bachelor’s degree in computer science or related fieldAgile tool