Senior Software Engineer – Ingestion Framework

This is a fully remote position, open to applicants in Canada.

📋 Description

• Develop and enhance the Python-based framework for metadata ingestion.

• Facilitate the ingestion of usage statistics, end-to-end lineage, schemas, operational metadata, quality signals, and business context.

• Design and implement connectors for Snowflake, Redshift, Kafka, Databricks, and other emerging infrastructures.

• Ensure that ingestion is cloud-native, scalable, resilient, observable, and easy to deploy and manage across customer environments.

• Utilize AI to create smarter connectors, automate metadata inference, enrichment, and intelligent ingestion workflows.

• Address distributed-systems challenges related to large-scale metadata extraction, incremental updates, reliability, and performance.

• Lead the technical direction for a small engineering team and provide mentorship to junior engineers.

• Enhance engineering quality, design, and execution standards.

• Collaborate with product, platform, and customer-facing teams to transform data integration requirements into product features.

⛳️ Requirements

• Over 8 years of experience in designing, building, and delivering production-level distributed backend systems.

• Strong expertise in Python, including ownership of features throughout the entire lifecycle from design to implementation, testing, deployment, and iteration.

• Excellent foundational knowledge in computer science, supported by a degree in Computer Science or equivalent practical experience.

• Profound understanding of API design, algorithmic complexity, reliability, and concepts related to distributed systems.

• Practical and ownership-driven mindset with the ability to navigate ambiguity and resolve complex technical challenges.

• Experience with modern data platforms such as Snowflake, Databricks, Kafka, or similar systems is highly regarded.

• Experience working across multiple cloud environments is a significant advantage.

• Prior experience in leading or mentoring a small team or engineering group is a strong plus.

🏝️ Benefits

• Equity for all: Each team member receives an ownership stake in the company.

• Remote work options available.

• Monthly coworking stipend provided.

• Comprehensive health coverage, including 99% of medical, dental, and vision premiums for employees and 65% for dependents.

• Flexible Spending Accounts (FSAs) available.

• Dependent Care FSA options.

• Carrot Fertility inclusive benefits and family-forming support for all U.S. employees.

• Unlimited PTO and sick leave policies.

• Engage in high-impact challenges at the core of enterprise AI infrastructure.

• Collaborate with a team of high-caliber professionals.

• Opportunities to develop next-generation AI-native data systems.