Anti-Bot Engineer

This is a fully remote position, open to applicants in Pakistan.

📋 Description

• Design, develop, and maintain extensive Python-based scraping frameworks aimed at highly secured websites (including those resembling Google environments).

• Create resilient extraction systems adept at managing dynamic, JavaScript-intensive pages through browser automation and hybrid techniques.

• Continuously modify systems to accommodate frequent alterations in page layouts, request patterns, and anti-bot strategies.

• Construct reliable, production-grade data extraction pipelines with a strong focus on accuracy, observability, and fault tolerance.

• Implement sophisticated strategies such as proxy rotation, fingerprinting, session management, and request routing to maintain stability at scale.

• Monitor system performance, proactively identify anomalies, and troubleshoot intricate production failures across distributed systems.

• Enhance scraping infrastructure for optimal performance, cost-effectiveness, and reliability at scale.

• Collaborate with data engineering and product teams to ensure the scraped data is structured, validated, and trustworthy.

• Manage and enhance systems operating continuously within cloud environments.

• Document system architecture, scraping methodologies, and operational procedures to ensure long-term maintainability.

• Contribute to the improvement of resilience, automation, and adaptability in challenging environments.

⛳️ Requirements

• Over 7 years of professional software engineering experience, concentrating on backend systems, data engineering, or distributed systems.

• Demonstrated experience in developing and managing large-scale production web scraping solutions.

• Extensive hands-on experience scraping Google or similarly complex, heavily secured/anti-bot environments.

• Strong proficiency in Python (or equivalent production languages such as Go, Rust, or JavaScript).

• Solid understanding of HTTP internals: headers, cookies, TLS, redirects, sessions, and browser networking behavior.

• Familiarity with browser automation frameworks like Playwright, Selenium, Puppeteer, or similar tools.

• In-depth knowledge of HTML parsing, DOM traversal, and high-performance data extraction methods.

• Proven experience in managing anti-bot systems, including rate limiting, CAPTCHAs, IP rotation, and fingerprinting.

• Experience in designing asynchronous and concurrent systems for high-throughput tasks.

• Exceptional debugging skills within distributed, failure-prone production environments.

• Experience operating cloud-based systems at scale with significant operational responsibility.

• Comfortable navigating highly adversarial and rapidly evolving technical landscapes.

🏝️ Benefits

• Fixed Shifts: 12:00 PM - 9:30 PM IST (Summer) | 1:00 PM - 10:30 PM IST (Winter)

• No Weekend Work: Genuine work-life balance, not just a slogan

• Day 1 Benefits: Laptop and comprehensive medical insurance provided

• Support That Matters: Mentorship, community, and forums for sharing ideas

• True Belonging: A long-term career with recognition for your contributions