Datadog - Vector.dev

We have an immediate long-term opportunity with one of our prime clients for a position Datadog - Vector.dev to work on Remote basis.

Must Haves:- 

• Vector.dev
• Experience: Proven experience in managing data pipelines, with a strong focus on Datadog and Vector.dev.
• Data Pipeline and Storage: Frohman detailed the process of collecting logs from various sources, structuring them through a pipeline using vectored dev for rate limiting and security
• Scale and Talent Needs: The platform aims to handle 100 petabytes a month initially, requiring sharp talent, especially vectored dev. They are seeking vendor support for architecture and operational expertise.

Job Description:

We are seeking a skilled Pipeline Support Engineer to join our team. The ideal candidate will be responsible for supporting and maintaining our data pipeline infrastructure, which leverages Datadog and Vector.dev. This role is crucial in ensuring the seamless integration and flow of data across our systems.

Key Responsibilities:

• Pipeline Management: Oversee the setup, configuration, and maintenance of data pipelines using Datadog and Vector.dev.
• Monitoring and Troubleshooting: Utilize Datadog to monitor pipeline performance, identify issues, and implement solutions to ensure optimal functionality.
• Data Integration: Work closely with data engineers and developers to integrate various data sources into the pipeline.
• Performance Optimization: Continuously optimize the pipeline for performance, scalability, and reliability.
• Documentation: Maintain comprehensive documentation of pipeline configurations, processes, and troubleshooting steps.
• Collaboration: Collaborate with cross-functional teams to understand data requirements and ensure the pipeline meets business needs.
• Support: Provide technical support and training to team members on pipeline-related issues and best practices.

Qualifications:

• Education: Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Experience: Proven experience in managing data pipelines, with a strong focus on Datadog and Vector.dev.
• Technical Skills: Proficiency in scripting languages (e.g., Python, Bash), and familiarity with cloud platforms (e.g., AWS, Azure).
• Problem-Solving: Strong analytical and problem-solving skills, with the ability to troubleshoot complex issues.
• Communication: Excellent verbal and written communication skills, with the ability to convey technical information to non-technical stakeholders.
• Team Player: Ability to work effectively in a collaborative team environment.