Senior Data Engineer

This is a fully remote position, open to applicants in North America.

📋 Description

• Design and manage essential forward- and reverse-ETL processes to provide data to relevant stakeholders.

• Create scalable patterns within the transformation layer to ensure consistent integrations with BI tools across multiple business sectors.

• Enhance and sustain the continuously evolving components of the Alpaca Data Lakehouse architecture.

• Work collaboratively with sales, marketing, product, and operations teams to meet critical data flow requirements.

• Administer the system and address production issues promptly.

⛳️ Requirements

• Over 7 years of experience in data engineering, with at least 2 years focused on building scalable, low-latency data platforms capable of managing over 100 million events per day.

• Expertise in at least one programming language, with a strong working knowledge of Python and SQL.

• Familiarity with cloud-native technologies such as Docker, Kubernetes, and Helm.

• Significant hands-on experience with relational database systems and object storage implementations, such as Apache Iceberg.

• Extensive hands-on experience with Google Cloud Platform and its various data-related services (e.g., Composer, Dataproc, Datastream).

• Experience in creating scalable transformation layers, ideally through formalized SQL models (e.g., dbt).

• Capability to thrive in a fast-paced environment and adjust solutions to meet evolving business requirements.

• Familiarity with ETL orchestrators/frameworks like Apache Airflow and Airbyte.

• Practical experience with streaming systems, such as Kafka.

• Exposure to infrastructure, DevOps, and Infrastructure as Code (IaaC) tools, such as Terraform.

• Comprehensive knowledge of distributed systems, storage, transactions, and query processing utilizing open-source distributed query engines like Trino (formerly PrestoSQL).

🏝️ Benefits

• Competitive Salary & Stock Options

• Health Benefits

• New Hire Home-Office Setup: One-time USD $500

• Monthly Stipend: USD $150 per month via a Brex Card