Senior Data Engineer

This is a fully remote position, open to applicants in Florida.

📋 Description

• Design and implement data architectures that are native to Snowflake.

• Spearhead the development and enhancement of contemporary data architectures on the Snowflake platform.

• Ensure that the architecture accommodates high-volume data workloads, meets near-real-time freshness demands, and integrates effortlessly with both upstream operational systems and downstream analytics users.

• Oversee the creation of intricate, end-to-end data pipelines utilizing native Snowflake services.

• Architect and construct data pipelines with Dynamic Tables for SQL-based transformations that are declarative, featuring automated dependency management and incremental refresh.

• Facilitate real-time and near-real-time data ingestion through Snowpipe and Snowpipe Streaming.

⛳️ Requirements

• A minimum of 7 years of experience in data engineering or related disciplines.

• Profound understanding of data engineering principles.

• Comprehensive knowledge of data modeling, including the design and maintenance of relational, dimensional, and semi-structured data models within Snowflake.

• Expertise in data warehousing concepts, with practical experience in designing multi-layer transformation pipelines (staging, intermediate, marts).

• Strong grasp of incremental processing patterns, change data capture (CDC), and slowly changing dimension (SCD) strategies.

• Proficiency in the Snowflake platform and its native data pipeline services.

• Advanced expertise with Snowflake Dynamic Tables (TARGET_LAG, REFRESH_MODE, pipeline dependency graphs, incremental versus full refresh).

• Hands-on experience with Snowpipe and Snowpipe Streaming for continuous and real-time data ingestion.

• Solid understanding of Snowflake Streams and Tasks for event-driven and procedural pipeline orchestration.

• Experience with Snowpark (Python/Scala) for complex data transformations and user-defined functions/user-defined table functions (UDFs/UDTFs).

• Familiarity with Snowflake Cortex AI functions to embed AI/ML capabilities into data pipelines.

• Proficiency in SQL, Python, and data engineering frameworks.

• Advanced SQL competencies, including window functions, common table expressions (CTEs), recursive queries, handling semi-structured data (VARIANT, OBJECT, ARRAY), and performance optimization.

• Proficient in Python for Snowpark development, automation scripting, and integration tasks.

• Experience in developing modular SQL transformations, including testing and documentation, adhering to native Snowflake patterns and shared engineering standards.

• Familiarity with orchestration platforms like Apache Airflow / Astronomer for pipeline scheduling and monitoring.

• Practical experience designing solutions for real-time and near-real-time data pipelines using Snowpipe Streaming and Kafka connectors.

• Strong knowledge of cloud infrastructure and cost management strategies.

• Expertise in cloud platforms (AWS, Azure, or GCP) with an emphasis on integration with Snowflake (external stages, storage integrations, PrivateLink).

• Experience in managing Snowflake costs, including warehouse sizing strategies, auto-suspend/resume features, resource monitors, and query optimization.

• Familiarity with infrastructure-as-code tools (Terraform, Pulumi) for declaratively managing Snowflake resources.

🏝️ Benefits

• A comprehensive benefits package that includes paid time off.