Senior Python Data Scraping Engineer, Freelance

This is a fully remote position, open to applicants in New York.

📋 Description

• Take ownership of comprehensive data extraction workflows from intricate websites.

• Provide complete, precise, and dependable structured datasets.

• Utilize tools such as Apify, OpenRouter, and custom workflows to enhance data collection, validation, and task execution.

• Extract information from dynamic and interactive web sources, including content rendered by JavaScript.

• Adjust scraping techniques in response to evolving site behavior and minor structural changes.

• Ensure data quality through validation checks, consistency controls across sources, formatting compliance, and systematic verification.

• Scale scraping processes for extensive datasets using efficient batching or parallelization methods.

• Monitor failures and uphold scraping stability.

⛳️ Requirements

• A minimum of 5+ years of relevant experience in data engineering, web scraping, automation, or software development.

• Strong expertise in Python web scraping, including tools like BeautifulSoup, Selenium, or similar, as well as handling dynamic content (JS, AJAX, infinite scroll) and APIs via proxies.

• Proven capability to extract data from complex structures, which include hierarchies, archived pages, and inconsistent HTML.

• Solid foundation in data cleaning, normalization, and validation, delivering structured datasets in formats such as CSV, JSON, or Google Sheets.

• Demonstrated experience in managing anti-bot mechanisms and dynamic site structures on a large scale.

• Familiarity with cloud infrastructure (AWS or equivalent) and containerization (Docker) in practical workflows.

• Direct experience with LLM frameworks such as LangChain or OpenRouter applied to automation tasks.

• Strong attention to detail and a commitment to data accuracy.

• Self-motivated work ethic and the ability to troubleshoot independently.

• Proficiency in English at an Upper-intermediate (B2) level or higher.

• A Bachelor's or Master's Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is an advantage.

• A GitHub link would be a plus.

🏝️ Benefits

• Freelance opportunity.

• Part-time remote work.

• Estimated 10–20 hours per week during active project phases.

• Compensation of up to $45 per hour equivalent, contingent on level and pace of contribution.