Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Define and advocate for the architectural roadmap and best practices related to end-to-end data pipelines, ensuring they are scalable, reliable, and secure.

• Mentor engineers, perform code reviews, and manage project timelines to ensure the swift delivery of data products.

• Collaborate with Data Scientists, Analysts, and business stakeholders to convert requirements into production-ready data solutions.

• Work alongside Data Scientists and ML Engineers on enhancing data accessibility, supporting model development, and ensuring data quality.

• Design, construct, and optimize high-volume data ingestion and transformation jobs utilizing dbt Core and AWS Glue.

• Develop and sustain modular, reusable data pipelines with orchestrators like Dagster.

• Manage and implement real-time data flows using Confluent platforms or AWS streaming services such as Kinesis.

• Enforce data anonymization practices and adhere to data security and privacy regulations.

• Implement and maintain the Iceberg open table format for effective schema evolution and data management.

• Enhance query performance and cost efficiency across Snowflake, Amazon Redshift, and AWS Athena.

• Integrate monitoring and observability into data pipelines using Splunk to identify bottlenecks and resolve production issues.

⛳️ Requirements

• Over 10 years of practical, progressive experience in Data Engineering, Data Architecture, or a related Full-Stack Data role.

• Profound conceptual understanding of core data engineering principles, ETL/ELT patterns, and metadata management.

• Demonstrable experience in building and managing petabyte-scale data infrastructures in a cloud-native setting.

• Experience in the insurance industry is preferred but not required.

• Proficiency with AWS services, including S3, IAM, and VPC.

• Familiarity with Talend, dbt Core, Iceberg, AWS Glue Catalog, Snowflake, Redshift, Athena, Splunk, AWS streaming services, and Git.

• Strong skills in SQL, PySpark, and Python.

• Understanding of DataOps and DevOps fundamentals.

🏝️ Benefits

• Annual bonus.

• Benefits information available at https://www.exlservice.com/us-careers-and-benefits.

• Remote work / work from home options.