Python Developer – Senior

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Design, develop, and enhance systems for data collection, processing, and integration using Python, ensuring scalability, resilience, and optimal performance.

• Lead technical projects from the design phase through implementation, testing, and production deployment.

• Create automated data collection solutions utilizing web scraping, web crawling, browser automation, API consumption, and integration with external sources.

• Construct large-scale data collection architectures, focusing on parallelism, concurrency, session management, and resource efficiency.

• Formulate strategies for data extraction from dynamic websites, JavaScript applications, and both structured and unstructured documents.

• Develop pipelines for the extraction, transformation, normalization, validation, and enrichment of substantial data volumes.

• Identify and address performance bottlenecks, enhancing throughput, latency, memory usage, and operational costs.

• Implement resilience features such as retries, timeouts, backoff, idempotency, and failure recovery.

• Advance the architecture of Python services by advocating for modularization, component reuse, and reduction of technical debt.

• Establish observability mechanisms to monitor source availability, success rates, execution times, and data quality.

• Investigate production incidents, determine root causes, and implement structural enhancements.

• Explore innovative data collection technologies and methodologies, including AI for intelligent information extraction and process automation.

• Utilize AI tools in development (AI First) to enhance productivity, quality, and delivery speed.

• Participate in architectural decision-making, conduct code reviews, and assist in the technical development of junior professionals.

⛳️ Requirements

• Strong proficiency in Python, including asynchronous programming, concurrency, memory management, and performance enhancement.

• Demonstrated experience in web scraping and web crawling with tools such as Scrapy, BeautifulSoup, Playwright, Selenium, or similar technologies.

• Deep understanding of HTTP/HTTPS, headers, cookies, sessions, authentication, and the workings of web applications.

• Experience in browser automation and data extraction from dynamic applications rendered in JavaScript.

• Solid knowledge of software architecture, design patterns, and the development of distributed, scalable systems in Python.

• Familiarity with frameworks like FastAPI, Django, and/or Flask.

• Experience processing large datasets using Pandas, Polars, or comparable tools.

• Understanding of asynchronous and parallel processing utilizing asyncio, multiprocessing, queues, and workers.

• Experience with messaging systems such as RabbitMQ, Kafka, SQS, Redis, or equivalent technologies.

• Proficiency with MongoDB and/or PostgreSQL, including data modeling, indexing, and query optimization.

• Capability to develop resilient data collection solutions featuring reprocessing, deduplication, execution control, and failure recovery.

• Experience in automated testing, observability, monitoring, and troubleshooting in production environments.

• Knowledge of Docker, Linux, and cloud infrastructures (AWS and/or GCP).

• Familiarity with AI development tools (AI First) and their integration into daily workflows.

• Ability to autonomously investigate complex challenges, propose solutions, and manage end-to-end delivery.

• Strong communication skills, analytical thinking, a sense of ownership, and the ability to work collaboratively with other developers.

• A university degree or technical diploma.

• Proficiency in English.

🏝️ Benefits

• Wellhub (Gympass)

• Language Allowance

• Pet Life and Dog Life

• English Pass

• Life Insurance

• Contractor arrangement (PJ)