Senior Site Reliability Engineer

About the job

Job Title Senior Site Reliability Engineer

Location Atlanta, Georgia (Hybrid Role 2-3 days onsite per week, rest remote)

Duration Long Term

Job Description

Required Skillset 

Manage and optimize data streaming and API components in OpenShift Onpremise and AWS.

Proactively review the application's APIs and processes to identify opportunities to optimize the response times for various application components.

Automate various types of testing including data quality checks, automate delivery to production and automate deployment for production

Develop integrations between the application in Onpremise and AWS and our third-party tools (ServiceNow, VersionOne, Sumo)

Work with teams to create SLI/SLO's

Actively monitor and lead troubleshooting of degraded performance and hard to define issues for the platform applications, develop the solution and document artifacts in the back log from root cause analysis.

Evolve the cloud infrastructure ecosystem for our application suite by experimenting with emerging technologies and completing prototypes to understand benefit

Design and develop CI/CD pipeline to deploy various application artifacts, including APIs and Data Process Jobs.

Analyze, design and develop the artifacts to configure the monitoring and alerting metrics so the support engineers can proactively and timely validate, troubleshoot and resolve the issues.

Maintain data integrity and access control by using AWS security tools and services such as HSM, IAM, etc.

Understand and develop tools to monitor AWS billing for the services, generate cost related reports and help develop and implement cost optimization strategies.

Work with enterprise security architects to design and implement data security tools, measures, data encryption, key management; design and develop solutions to address the security vulnerabilities discovered by internal security audit team, as well as by the vendors, security community, etc.; design and develop solutions for support team to regularly scan and review to fix security issues

Regularly and proactively monitor and analyze the capacity and performance of the platform, work with architecture team to design and implement elastic infrastructure to accommodate the irregular burst of user traffic/requests.

Work with architecture team to develop backup strategy and implement the backup solution for critical data and application components for service restoration and disaster recovery purpose.

Work with architecture, infrastructure, and application teams to provide input on continuous improvement on the design, performance and security enhancements.

Desired Skillset

Deep understanding of the operations of AWS cloud platforms.

Must be well versed in the automation, scripting, monitoring, including use of tools from the major cloud platforms, including but not limited to OpenShift Cloud Formation, Terraform, Ansible, Shell, Python

Preferable for candidates with significant technical knowledge with infrastructure layers, including but not limited to: Linux OS, major virtualization platforms, Traditional and software defined network, Load Balancers, firewall, API tools, element/performance/intelligent monitoring tools, storage, backup strategy, etc.

Significant knowledge and experience in end to end operations for enterprise systems and applications, including driving issue resolution for mission critical systems.

Must have experience working to automate, operationalize and improve the Development/QA using CI/CD tools (Gitlab, Github, Jenkins, Maven, Gradle, Nexus)

Working experience with Software Release Management.

Desired Qualification 

BS degree in Computer Science or a related technical field or equivalent practical experience.

Minimum Experience

3+ years of related DevOps, SysOps engineering experience with focus on major cloud platforms (AWS preferred).

2+ years of application development experience including data streaming, deploying/monitoring high availability critical application components.

1+ Years in Site Reliability Engineering organization preferred

Overall 12+years of experience

Responsibilities

As a lead engineer with Retail, Site Reliability Engineering team, you will be at the forefront of Cloud and Big Data technology. In this role you will establish yourself as a technical leader by exposing yourself to a broad range of industry leading technologies that will help to drive acceleration. The ideal candidate will have expert design and development capabilities and be positioned to contribute to a growing set of services and features for the ecosystem. This role will be supporting highly available, business critical applications. This role will serve as the escalation point for complex and hard to define issues in both on premise and AWS environments. We are seeking talented engineers, well versed in DevOps technologies, automation, infrastructure orchestration, configuration management, continuous integration, troubleshooting of complex issues, who are not constrained by how "things are usually done".

What you'll be doing? 

Engage in and improve the whole lifecycle of services-from inception and design through deployment, operation, and refinement

Support capacity planning, availability, scalability, security and latency considerations for new infrastructure and service provisioning as appropriate

Responsible for improvements to end-to-end availability and performance of mission critical services and build automation to prevent problem recurrence.

Partner with business and technical product owners to set SLOs / SLIs / error budgets to manage reliability of infrastructure and applications

Partner with other SREs to bring best practices or learnings from across the organization to them

Scale and optimize existing infrastructure and services sustainably through mechanisms, including automation, and evolve them by improving reliability and efficiency

Manage end-to-end availability and performance of mission-critical services and build automation to prevent problem recurrence

Maintain infrastructure (infrastructure as code) and services by measuring, and monitoring system metrics to proactively identify operational efficiencies, potential outages and security threats in Development, UAT, Staging and Production environments

Practice sustainable incident response and blameless postmortems

Build infrastructure and drive projects that break things with the aim to improve the robustness of production systems

Use the core Site Reliability Engineering principles of change management, monitoring, emergency response, capacity planning, and production readiness reviews to run the platform

Step back to observe patterns and develop innovative tools and automation to eliminate or minimize menial tasks. Use those learnings to drive the best operational practices

Develop and maintain solution and operational documentation and designs for all infrastructure and services within the scope of SRE Preserve operational visibility and response capabilities - fixing and improving our dashboards, alerts, and automation

Maintain operational uptime and reliability by participating in triage and issue support calls for mission critical systems