Senior Software Engineer – Crawler

This is a fully remote position, open to applicants in Washington.

📋 Description

• Develop and deploy components for scalable, fault-tolerant web crawling and data extraction pipelines.

• Produce clean, production-ready code in Java and Python.

• Create and manage ETL/ELT pipelines for extensive data extraction and transformation.

• Utilize cloud infrastructure on GCP and AWS, focusing primarily on GKE.

• Enhance observability, reliability, and operational excellence across various systems.

• Collaborate with product and data science teams to deliver meaningful solutions.

• Engage in code reviews, documentation, and knowledge sharing within the team.

• Keep updated with the latest web technologies, anti-crawling techniques, and AI-driven extraction methods.

• Take ownership of features from design through to deployment and ongoing operation.

• Make architectural decisions at the component level.

• Contribute to ZoomInfo's data platform infrastructure that supports business intelligence products.

⛳️ Requirements

• Over 5 years of professional software engineering experience in building production systems.

• Strong foundation in computer science principles, including algorithms, data structures, concurrency, and distributed systems.

• Proficient in Java and/or Python.

• Proven history of owning features throughout the entire process from design to deployment and operation.

• Capable of making informed architectural decisions at the component level.

• Practical experience with cloud data warehouses like BigQuery or Snowflake.

• Background in designing and managing large-scale ETL/ELT pipelines.

• Familiarity with orchestration tools such as Apache Airflow.

• Experience with streaming or event-driven systems, such as Apache Kafka.

• Production experience on cloud platforms like GCP or AWS.

• Hands-on knowledge of Kubernetes, including GKE/EKS, for distributed workloads.

• Knowledge of infrastructure-as-code tools such as Terraform.

• Excellent communication skills with the ability to articulate technical decisions clearly.

• Ability to navigate ambiguity and adapt quickly.

• Pragmatic problem-solver with a proactive approach.

• Self-motivated individual comfortable in fast-paced, dynamic environments.

🏝️ Benefits

• Comprehensive benefits package.

• Programs focused on holistic mind, body, and lifestyle well-being.

• Additional compensation options, including Bonus, Commission, Equity, and other benefits may also be available.