Senior Data Engineer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Design and manage production data pipelines within a structured bronze/silver/gold data platform.

• Source, clean, and transform large-scale data efficiently.

• Address challenges related to incremental synchronization and stream processing as data volumes and account numbers increase.

• Develop regional data architectures that comply with data residency and PII regulations.

• Create production systems, orchestration, and deployment strategies for Data Science models and analyses.

• Establish robust, monitored, and reliable pipelines for machine learning and analytical purposes.

• Take ownership of pipeline infrastructure, orchestration, deployment mechanisms, observability, scalability, and operational dependability.

• Collaborate with Data Science teams when production issues affect both model and platform.

• Function as an individual contributor without management responsibilities.

⛳️ Requirements

• Over 5 years of experience designing and operating data pipelines in large-scale production environments.

• Proficiency in batch and stream data processing, including incremental and streaming architectures.

• Strong expertise in Python and adherence to production-level software engineering standards, including testing, code reviews, version control, and monitoring.

• Proven proficiency in SQL.

• Experience in designing and orchestrating ETL/ELT pipelines.

• Familiarity with regional and multi-region data storage and data residency requirements.

• Experience with parallel data frameworks such as Dask, Spark, or similar technologies.

• Background in API design and implementation, including microservices and RESTful architecture.

• Experience working in cloud environments such as GCP or AWS.

• Knowledge of Docker/containers and Kubernetes.

• Experience in deploying or serving machine learning models.

🏝️ Benefits

• Remote work opportunity within the US.

• Minimal travel requirements.

• Limited physical demands.

• Chance to take ownership of platform challenges related to ML and analytics scalability.

• Engaging scaling challenges including incremental/streaming processing, regional data residency, and managing petabyte-scale data.

• Work within a small, senior team with genuine ownership and visibility to leadership.

• Commitment to an equal opportunity and inclusive work environment.