AI Automation Lead

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and manage the platform for crawling, data ingestion, normalization, and matching.

• Process static and JavaScript-rendered websites, PDFs, pagination, and multilingual content.

• Ensure complete source provenance for every record.

• Create the canonical data model for products, suppliers, and offers.

• Develop entity-resolution logic to accurately match listings with real parts.

• Utilize LLMs for extraction, translation, and classification while implementing evaluation, confidence thresholds, and human review processes.

• Integrate the platform with Zoho CRM's RFQ workflow and subsequently with QuickBooks.

• Collaborate with Sales, Purchasing, Operations, and Finance to uncover AI and automation opportunities.

• Transform opportunities into actionable pilots, train staff, and monitor impact.

• Deliver a functional architecture and data model within the first 90 days.

• Integrate at least 10 sources with complete provenance.

• Provide searchable and normalized product and supplier data.

• Assess matching accuracy and establish source-health monitoring protocols.

• Implement an initial integration with Zoho RFQ.

• Draft a documented strategy for scaling to hundreds of sources.

⛳️ Requirements

• A minimum of five years of direct software/data engineering experience, with real production ownership of web crawling or large-scale data acquisition systems — excluding personal projects or academic coursework.

• Proficiency in advanced Python and strong SQL skills.

• Familiarity with a production crawling framework (e.g., Scrapy) and browser automation tools (Playwright or Selenium) for JavaScript-rendered sites.

• Comprehensive understanding of HTTP, rate limiting, retries, queues, and failure recovery mechanisms.

• Experience in building normalized pipelines from disorganized, conflicting data, alongside entity resolution, fuzzy matching, and deduplication tasks supported by actual accuracy metrics.

• Strong skills in PostgreSQL (or similar databases), Docker, and at least one major cloud platform.

• Proficient in utilizing LLM APIs in a production environment — focusing on structured outputs, evaluation, and cost management — without viewing them as substitutes for deterministic checks.

• Proven experience in implementing an AI or automation solution within a real organization, preferably across multiple departments.

• Skilled in conducting training and translating business requirements into technical specifications for other engineers.

• Nice to have: Experience with Temporal, Dagster, Prefect, or Airflow.

• Nice to have: Familiarity with OpenSearch or Elasticsearch.

• Nice to have: Knowledge of RapidFuzz, Splink, or embeddings-based matching techniques.

• Nice to have: Experience with Zoho, Salesforce, HubSpot, QuickBooks, or other ERP systems.

• Nice to have: Background in electronic components or other intricate technical catalogs.

• Nice to have: Previous experience as the first or lead engineer on a new platform.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health insurance plans.

• Flexible work hours and remote work options.

• Opportunities for professional development and training.

• A collaborative and innovative work environment.