Data Engineer – Cloud, SaaS Integrations

This is a fully remote position, open to applicants in United Kingdom, +5 more countries.

📋 Description

• Develop new integrations with third-party billing and usage APIs to enhance vendor coverage.

• Maintain current integrations as vendor APIs and pricing structures evolve.

• Minimize the marginal costs associated with creating subsequent integrations.

• Utilize AI throughout the workflow for codebase and API exploration, prototyping, code generation and review, debugging, testing, and documentation.

• Oversee data quality, addressing issues such as duplication, ingestion and reprocessing gaps, race conditions, marketplace spend deduplication, and support for negotiated rates.

• Establish checks and reconciliation processes to ensure data accuracy.

• Design and implement normalization models for units, currencies, time granularity, resource and service taxonomies, and cost categories.

• Manage orchestration, scheduling, and backfill mechanisms for ingestion pipelines from start to finish.

• Make backfills routine, safe, and self-service.

• Collaborate with product, support, and engineering teams to identify and resolve data gaps.

• Independently propose necessary work instead of waiting for specifications.

⛳️ Requirements

• 3+ years of professional experience in data engineering or a data-intensive backend role, with production ownership of pipelines relied upon by others.

• Proficient SQL skills, including the ability to write, read, and reason about the performance of complex analytical queries.

• Practical experience in building and operating data pipelines using an orchestration framework.

• Experience with Dagster or Airflow is highly desired; equivalent experience with Prefect, dbt, or a similar tool is relevant if willing to work with Dagster/Airflow.

• Strong proficiency in Python or another language commonly used for data tasks.

• Familiarity with a cloud data warehouse or analytical store such as BigQuery, ClickHouse, Snowflake, Redshift, or similar.

• Experience integrating third-party REST APIs, including handling pagination, rate limits, partial failures, late-arriving and restated data, and dealing with inaccurate vendor documentation.

• A keen sense for data accuracy, reconciliation, and assertions.

• An AI-augmented working style and experience utilizing AI tools throughout an engineering workflow.

• Experience in developing solutions in the cloud and/or using cloud services.

• Excellent written and verbal communication skills in English.

• Self-organized, goal-oriented, self-motivated, confident, thorough, and tenacious.

• A BA/BS degree or equivalent practical experience is considered an asset.

🏝️ Benefits

• Unlimited Vacation

• Flexible Working Options

• Health Insurance

• Parental Leave

• Employee Stock Option Plan

• Home Office Allowance

• Professional Development Stipend

• Peer Recognition Program

• Full-time employee benefits