Senior Scraping Engineer – Web Scraping, Anti-bot

This is a fully remote position, open to applicants in Spain.

📋 Description

• Design and build architectural frameworks.

• Data-plane components: high-parallel crawling, session management, anti-bot evasion, CAPTCHA solving.

• Control-plane features: API development, user authorization, billing systems, quota management, and monitoring solutions.

• Infrastructure management: utilizing microservices, Docker/Kubernetes, CI/CD practices, and observability tools.

• Take ownership of the development process.

• Define and implement development and release protocols.

• Oversee code quality and make technical decisions.

• Conduct anti-bot research and develop bypass methods.

• Fingerprint techniques at both browser and network levels (JA3, TLS, CDP).

• Bypass protections from services such as Cloudflare, Akamai, Qrator, ServicePipe, Variti, and others.

• Engage in DevOps and Site Reliability Engineering (SRE) practices.

• Set up CI/CD pipelines, Helm charts, and configure scaling options.

• Organize monitoring using tools like Prometheus, Grafana, and Loki, along with managing SLA processes.

⛳️ Requirements

• 4-6+ years of experience in backend development, data engineering, automation, or high-load systems.

• Proficient in web scraping, particularly with protected, dynamic, or JavaScript-heavy websites.

• Familiarity with Python asynchronous programming; experience with Go is a plus.

• Proficient in tools such as Playwright, Puppeteer, Selenium, Scrapy, browser automation, and CDP.

• Strong grasp of HTTP, TLS, HTTP/2, cookies, sessions, headers, and proxies.

• Experience with Redis, message queues/brokers, Docker, CI/CD, and observability practices.

• Kubernetes experience is preferred; extensive K8s/SRE knowledge is an advantage but not a strict requirement within this budget.

• Capable of producing clear technical documentation in English.

• Possess an ownership mindset focusing on reliability, monitoring, incident resolution, and cost management.

🏝️ Benefits

• Contribute to the development of the company's flagship product from the ground up.

• Enjoy a high degree of autonomy and influence over architectural and team decisions.

• Access to a budget for conferences, training, and essential work tools.

• Attractive compensation package.

• Flexibility to work remotely.