Web Scraping Lead – Global Team

This is a fully remote position, open to applicants in India, +1 more country.

📋 Description

• Lead, mentor, and develop a team of web scraping engineers.

• Take ownership of the architecture, roadmap, scalability, and reliability of scraping systems.

• Oversee the development and maintenance of scrapers utilizing Playwright, Puppeteer, Selenium, and other similar tools.

• Address intricate challenges involving dynamic websites, browser automation, fingerprinting, sessions, cookies, headers, and proxies.

• Implement monitoring, logging, alerting, and data quality practices.

• Propel enhancements in performance, reliability, and infrastructure.

• Collaborate with Engineering, Data, Analytics, and Product teams to prioritize requirements and deliver high-quality data.

• Engage in hiring, technical reviews, and engineering planning.

• Direct the design, scalability, reliability, and performance of scraping infrastructure.

⛳️ Requirements

• Over 7 years of software engineering experience, particularly in web scraping.

• Prior experience in engineering management or team leadership is preferred.

• Extensive hands-on experience with Playwright, Puppeteer, or Selenium.

• Profound understanding of HTTP, REST APIs, HTML/DOM, browser rendering, and TLS/SSL.

• Significant experience with fingerprinting, anti-bot challenges, cookies, headers, sessions, and proxy rotation.

• Ability to build scalable and reliable scraping infrastructure.

• Strong knowledge of monitoring, logging, metrics, and troubleshooting in production environments.

• Excellent communication and stakeholder management skills.

• Demonstrated ownership, leadership, and mentoring capabilities.

• Must not currently or in the future require visa sponsorship.

🏝️ Benefits

• Competitive salary and comprehensive benefits package.

• Clear career progression paths for both managerial and individual contributor roles.

• Regular career discussions and opportunities for internal mobility.

• Flexible working hours.

• Hybrid work environment.

• Unlimited vacation policy.

• Generous 401K matching program.

• Parental leave.

• Team-building events.

• Wellness budget.

• Learning reimbursement.

• Monthly work-from-home budget for remote employees.

• Health plans with no monthly out-of-pocket premiums.

• Dental and vision coverage.

• Commuter benefits.

• Opportunities for learning, self-improvement, and skill mastery.