Software Engineer, Data Acquisition

This is a fully remote position, open to applicants in Canada, +1 more country.

📋 Description

• Develop and sustain production-grade web scraping infrastructures, primarily utilizing Rust.

• Manage and supervise diverse distributed systems across various cloud providers, bare metal, and different operating systems.

• Troubleshoot production challenges using raw logs and telemetry data.

• Create and execute proxy management, rotation strategies, and network-layer evasion methods.

• Build fleet health monitoring systems, automated alerting mechanisms, and self-healing infrastructures.

• Partner with analysts and researchers to establish data requirements, integrate sources, and ensure data quality and freshness.

• Reverse-engineer web applications and APIs through network traffic inspection, JavaScript deobfuscation, and adapting to changes on target sites.

• Enhance system reliability, throughput, and maintainability via refactoring, optimization, and automation.

• Deploy and manage agentic workflows for data source discovery, onboarding, and troubleshooting.

⛳️ Requirements

• 2–5 years of professional experience in software engineering with a solid foundation in systems or backend development.

• Proficiency in at least one major programming language (such as Go, Python, C++, Java, or similar).

• In-depth knowledge of networking fundamentals including TCP/IP, DNS, HTTP internals, CDNs, proxies, load balancing, and queue management.

• Practical experience with web scraping or large-scale data acquisition.

• Familiarity with anti-bot/anti-automation countermeasures and evasion techniques.

• Experience in managing heterogeneous distributed infrastructures across various operating systems, hardware, and hosting providers.

• Strong skills in log analysis and monitoring.

• Hands-on experience in building or operating agentic workflows utilizing LLM-based agents.

• Solid networking intuition, covering TCP connections, DNS resolution, HTTP headers, and proxy chains.

• Experience with Rust is a significant advantage but not essential.

• Proficiency in Python is a plus.

• Experience with AWS, GCP, or DigitalOcean is advantageous.

• Background in high-throughput HTTP, proxy management, VPN/routing optimization, and connection pooling is beneficial.

• Exposure to real-time or latency-sensitive systems is a plus.

🏝️ Benefits

• Remote-first, distributed team with regular in-person sprints and retreats.

• Competitive total compensation with significant bonus potential linked to performance.

• Hardware and home-office stipend.

• Budget for conferences and learning opportunities.

• Comprehensive health benefits including medical, dental, and vision coverage.

• Life insurance provided.

• Open vacation policy.

• Flexible working hours and location options.

• 16 weeks of fully-paid parental leave along with support for returning to work.

• Retirement matching offered.