Senior Software Engineer, Data Ingestion

atHorizon3.aiRemoteUS flagUnited StatesFull-timeData EngineerSenior$199.8k – $260k/year

Posted 2 days ago

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, implement, and manage data ingestion pipelines that process pentest results at scale.

• Handle scan output, identified assets, vulnerability findings, and network topology data.

• Take ownership of the full lifecycle delivery of pipeline features, including schema design, implementation, testing, deployment, and monitoring.

• Develop and maintain ETL components using Go and/or Python.

• Standardize diverse inputs from NodeZero execution engines.

• Collaborate with Asset Inventory and Data Platform teams to define and refine data contracts.

• Troubleshoot and resolve production issues related to pipeline latency, data accuracy, and throughput.

• Contribute to code and design evaluations.

• Provide engineering mentorship to P3 engineers.

• Report to the Data Ingestion Engineering Manager and work alongside backend engineers.


⛳️ Requirements

• Over 5 years of professional experience in software engineering with production backend systems.

• Proficient in Go and/or Python.

• Experience in building and maintaining production data pipelines or ETL systems.

• Familiarity with message queues or event streaming technologies such as Kafka or RabbitMQ.

• Experience with relational databases like PostgreSQL, focusing on schema design, query optimization, and migrations.

• Understanding of graph databases or graph data models.

• Knowledge of data normalization trade-offs, schema evolution, and backward compatibility.

• Experience deploying services on Kubernetes and within containerized environments using Docker.

• Comfortable with deploying and debugging distributed systems in AWS or similar cloud environments.

• Capable of instrumenting code, creating runbooks, and overseeing services throughout their entire lifecycle.

• Ability to establish data contracts with downstream consumers.

• Excellent written communication skills for design documents, pull request descriptions, and asynchronous updates.

• Experience in cybersecurity, security tools, vulnerability or threat data, graph databases, Apache Kafka, offensive security tools, or pentesting workflows is advantageous.

• Must be legally authorized to work in the United States.

• A bachelor’s degree or equivalent educational credential is not strictly required.


🏝️ Benefits

• Equity package in the form of stock options for all full-time positions.

• Health, vision, and dental coverage for you and your family.

• Flexible vacation policy.

• Generous parental leave.

• Opportunities for career development.

• An inclusive and collaborative workplace culture.

• Remote and hybrid work options based on role and location.

• Up to 5% travel may be necessary.

People also viewed

Karat7 hours ago

Senior Data Engineer

IN flagIndia OnlyFull-timeData Engineer
ApplyView job
Compass12 hours ago

Data Engineer

BR flagBrazil OnlyFull-timeData Engineer
ApplyView job
CI&T19 hours ago

Senior Data Engineer – Databricks

BR flagBrazil OnlyFull-timeData Engineer
ApplyView job
ELEKS19 hours ago

Data Quality Engineer – Energy Domain

UA flagUkraine, +2 more countriesFreelanceData Engineer
ApplyView job
GFT Technologies19 hours ago

Senior Data Engineer, COBOL

BR flagBrazil OnlyFull-timeData Engineer
ApplyView job
AWISEE19 hours ago

Senior Data Infrastructure Engineer – Scraping, Scale

SE flagSweden OnlyFull-timeData Engineer
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers