Senior Anti-Bot, Scraping Engineer

This is a fully remote position, open to applicants in Europe.

📋 Description

• Develop and sustain extensive Python scraping systems for Google and other well-protected websites.

• Create scrapers for dynamic, JavaScript-intensive pages utilizing browser automation and hybrid techniques.

• Adjust to modifications in page layouts, request processes, and anti-automation strategies.

• Construct extraction pipelines that prioritize data accuracy, consistency, and observability.

• Implement strategies for proxies, fingerprinting, and request routing to enhance stability and throughput.

• Oversee scraping health, identify failures promptly, and troubleshoot production problems.

• Enhance performance, cost-efficiency, and latency across scraping infrastructure.

• Collaborate with data engineers, data scientists, and product teams to ensure the usability and trustworthiness of scraped data.

• Document scraping methodologies and operational procedures.

• Report directly to the Head of Engineering.

• Engage in a two-stage interview process.

⛳️ Requirements

• Robust professional experience with Python or another primary language such as GoLang, Rust, or JavaScript, particularly in production-grade scraping systems.

• Extensive hands-on experience scraping Google or comparably secured platforms.

• Comprehensive understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behavior.

• Familiarity with Playwright, Selenium, Puppeteer, or similar browser automation frameworks.

• In-depth knowledge of HTML parsing, DOM navigation, and data extraction methods.

• Experience managing rate limiting, CAPTCHA, IP rotation, and bot detection systems.

• Proficient in working with asynchronous and concurrent scraping architectures.

• Experience operating scrapers at scale in cloud environments.

• Strong debugging abilities and capacity to analyze complex failure modes.

• Familiarity with headless browser fingerprinting and techniques for evading bot detection.

• Applicants must reside in one of the following countries: Spain, Italy, Portugal, Romania, Hungary, Czech Republic, Poland, Serbia, Lithuania, Bulgaria, Estonia, Latvia, Cyprus, or Greece.

• Capability to work in the UK timezone.

• Must have the right to work without visa restrictions in the current location.

🏝️ Benefits

• Stock options.

• Trust-based vacation policy: take as much time off as necessary.

• Remote-first workplace.

• Flexible working hours with autonomy over your workday.

• Customized training and career development through “Sherpa plans.”

• Monthly hack days and training sessions.

• Up to 6 months of fully paid maternity leave.

• 2 months of paternity leave.

• $200 stipend for home office expenses.

• $70 credit for a welcome gift.

• Day off for your birthday.

• Summer and Christmas parties.

• Annual and quarterly offsite events.

• Monthly meet-ups.

• Paid volunteer time each quarter.