Senior Datadog Security – Observability Engineer

This is a fully remote position, open to applicants in California, +1 more state.

📋 Description

• Take ownership and continuously enhance Datadog Cloud SIEM, as well as security monitoring and observability functions across production and corporate settings.

• Design, develop, and sustain detection and telemetry functionalities across Datadog, SentinelOne, Wiz, and related security platforms.

• Create, test, and refine high-fidelity Datadog detection rules that align with real-world attack scenarios and adversarial behaviors.

• Enhance alert quality by minimizing false positives, eliminating noise, and boosting detection accuracy.

• Design and manage Datadog log pipelines, processors, parsing rules, facets, indexes, archives, and retention strategies.

• Implement and develop detection-as-code practices for scalable, version-controlled, and testable rule management.

• Establish and enforce standards for logging, telemetry, and instrumentation across cloud infrastructure, applications, endpoints, and identity systems.

• Construct and optimize workflows for log ingestion, parsing, normalization, enrichment, and routing.

• Automate the onboarding of new telemetry sources while enhancing visibility across production and corporate environments.

• Correlate signals across Datadog, EDR, cloud, identity, and security platforms to enhance detection depth and the quality of investigations.

• Collaborate with Security Operations to refine triage workflows, incident response readiness, and escalation quality.

• Develop Datadog dashboards, monitors, analytics, and reporting tools that support operational decision-making across Security, SRE, and Engineering.

• Map and maintain detection coverage against the MITRE ATT&CK framework, identifying telemetry and detection gaps.

• Conduct detection gap assessments and adapt use cases based on threat intelligence, threat hunting, and emerging risks.

• Partner with cloud, infrastructure, product, and compliance teams to enhance secure logging and observability patterns throughout the software development lifecycle.

• Utilize AI-assisted tools such as Claude, ChatGPT, or similar platforms to aid in query development, detection engineering, investigations, automation, and technical documentation.

⛳️ Requirements

• A minimum of 5 years of experience in detection engineering, SIEM engineering, security engineering, security observability, or a related technical role.

• Extensive hands-on experience administering and engineering Datadog within complex cloud environments.

• Strong background in Datadog Cloud SIEM, Log Management, Security Monitoring, dashboards, monitors, and alerting.

• Proficient in designing and maintaining Datadog log pipelines, processors, parsing rules, facets, indexes, and retention strategies.

• Experience in building, testing, and tuning detection rules, correlation logic, and investigation workflows in Datadog.

• In-depth understanding of security telemetry across cloud, endpoint, identity, and application environments.

• Familiarity with log parsing, normalization, enrichment, and pipeline management.

• Strong knowledge of AWS and cloud-native infrastructure.

• Proficient in scripting or automation using Python, PowerShell, or similar languages.

• Experience utilizing Datadog APIs, Terraform, or similar infrastructure-as-code tools for automating configuration and platform management.

• Solid understanding of modern detection strategies, attacker behaviors, and the MITRE ATT&CK framework.

• Ability to troubleshoot complex issues across logs, metrics, traces, infrastructure, and application telemetry.

• Excellent communication skills with the ability to collaborate effectively across Security Operations, Engineering, Infrastructure, and SRE teams.

• Willingness to effectively utilize AI-assisted tools to enhance query development, detection analysis, troubleshooting, automation, and documentation.

• Preferred: Experience with SentinelOne, Wiz, or other related cloud and endpoint security platforms.

• Preferred: Experience with Datadog Application Performance Monitoring, Infrastructure Monitoring, distributed tracing, or synthetic monitoring.

• Preferred: Experience optimizing Datadog ingestion volume, indexing, retention, and platform costs.

• Preferred: Experience with SOAR, workflow automation, or response orchestration.

• Preferred: Familiarity with Sigma or other detection-as-code frameworks.

• Preferred: Experience operating Datadog in large-scale, multi-account, or multi-region AWS environments.

• Preferred: Background in high-scale SaaS, cloud-native, or security product environments.

• Preferred: Familiarity with zero-trust architectures, identity-centric security, and privileged access management.

• Preferred: Bachelor’s degree in Computer Science, Engineering, or a related field.

🏝️ Benefits

• Medical, Dental & Vision coverage (including domestic partnerships).

• Employer-paid Life Insurance & Supplemental life for Employee/Spouse/Child.

• Voluntary Short/Long Term Disability Insurance.

• 401K options (Roth/Traditional).

• A generous Paid Time Off (PTO) plan that acknowledges your commitment and tenure (including paid Bereavement/Jury Duty, etc.).

• Competitive annual bonuses above market standards.