Senior Data Engineer

This is a fully remote position, open to applicants in Connecticut.

📋 Description

• Design and develop scalable batch and streaming data pipelines utilizing Apache Spark, Kafka, and Flink.

• Construct and manage Bronze/Silver/Gold medallion architecture within the enterprise lakehouse using Delta Lake or Iceberg.

• Create and refine complex SQL and PySpark transformations for extensive datasets.

• Incorporate structured, semi-structured, and unstructured data sources into the lakehouse.

• Collaborate with data architects to advance physical and logical data models.

• Execute data quality checks and monitoring through Great Expectations or dbt tests.

• Develop Infrastructure-as-Code for pipeline environments using Terraform and Helm.

• Engage in code reviews and uphold engineering standards and best practices.

• Diagnose pipeline failures, performance bottlenecks, and data incidents.

• Mentor junior and mid-level data engineers while contributing to internal knowledge sharing.

⛳️ Requirements

• Over 6 years of data engineering experience with a proven track record in enterprise-scale delivery.

• Advanced proficiency in Python and SQL.

• Experience with PySpark is mandatory.

• Practical experience with Apache Spark, Delta Lake, or Apache Iceberg.

• Familiarity with Apache Airflow, Prefect, or Dagster.

• Strong understanding of AWS Glue, Azure Data Factory, or GCP Dataflow.

• Proficient with Git, CI/CD pipelines, Docker, and Kubernetes.

• Experience with dbt for managing transformation layers.

• Bachelor’s degree in Computer Science, Engineering, or a related technical field.

• Preferred: Experience with Databricks, Snowflake, or Apache Hudi.

• Preferred: Knowledge of streaming architectures and Apache Kafka.

• Preferred: Databricks Certified Data Engineer or AWS Data Analytics Specialty certifications.

🏝️ Benefits

• A discretionary incentive program may be offered as part of the compensation package.

• Comprehensive medical and other benefits based on full-time employment status.

• A collaborative and inclusive culture that emphasizes expertise, curiosity, and ownership.

• Opportunities for professional growth.

• Global Power Your Career conference for employees to network, learn, and develop.

• Experiential High-Potential Program providing professional development opportunities.

• Leadership Challenges offering skills and insights for managers and aspiring leaders.

• Accommodations available upon request for all aspects of the selection process.