Senior Data Engineer – Data Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, construct, and manage comprehensive data pipelines (both batch and near-real-time) that collect, transform, and deliver data from various sources to the enterprise data platform.

• Create and sustain curated data models, marts, and shared datasets in Snowflake and PostgreSQL that fulfill performance, quality, and access-control standards for multiple internal stakeholders.

• Establish data quality frameworks that encompass automated validation, schema enforcement, reconciliation checks, duplicate detection, and exception reporting, complete with clear audit trails.

• Collaborate with domain teams (such as Asset Management, Finance, and Operations) to comprehend data requirements, define contracts and SLAs, and provide platform capabilities that minimize custom engineering and manual labor.

• Develop parameterized, reusable pipeline components and templates that standardize ingestion patterns, transformations, and deployments throughout the platform.

• Set up and maintain data lineage, metadata, and documentation so that stakeholders can confidently trace data from its source to its final consumption.

• Work together with IT and security to implement role-based access controls, data masking, encryption, and compliance obligations across platform resources.

• Oversee pipeline orchestration, scheduling, dependency management, and alerting using workflow tools (such as Airflow) to guarantee reliable and recoverable execution.

• Enhance platform observability through logging, metrics, SLA monitoring, and incident response strategies that reduce downtime and data freshness issues.

• Support CI/CD and infrastructure-as-code methodologies for data platform assets, including version control, automated testing, and secure promotion across various environments.

• Assess and incorporate new platform technologies and methodologies (such as streaming, CDC, and data mesh principles) that improve scalability, cost-effectiveness, or time-to-value.

• Mentor junior engineers and contribute to platform standards, code review processes, and technical design documentation.

⛳️ Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative discipline.

• 5+ years of experience in data engineering or platform engineering, ideally within the financial services or regulated sectors (e.g., asset management, banking, insurance, fintech).

• Proficient in SQL and Python, with a proven history of developing production-quality data pipelines, transformations, and validation frameworks.

• Skilled in utilizing AI-assisted development tools to design, construct, and refine data pipelines while upholding code quality, security, and governance standards.

• Direct experience with Snowflake and PostgreSQL, encompassing performance tuning, cost optimization, and secure multi-tenant data access strategies.

• Familiarity with pipeline orchestration and workflow management tools (e.g., Apache Airflow, Dagster, or similar).

• Competent in Git, code review, and CI/CD methodologies for data platform development.

• Experience in designing dimensional or domain-oriented data models and delivering curated datasets for analytical and operational applications.

• Knowledge of data quality, lineage, and governance tools and practices is preferred.

• Familiarity with cloud data services (e.g., AWS, Azure, or GCP) and infrastructure-as-code (e.g., Terraform) is strongly preferred.

• Experience with streaming or change-data-capture (CDC) methods and event-driven architectures is advantageous.

• Understanding of financial data domains (such as portfolio, investor reporting, accounting) is beneficial but not mandatory; curiosity and the ability to engage with domain experts is crucial.

• Excellent communication and collaboration abilities; capable of transforming ambiguous requirements into well-defined technical designs and clear status updates.

• Knowledge of containerization (e.g., Docker/Kubernetes) and API/integration patterns for data services is a plus.

🏝️ Benefits

• Health insurance

• Retirement plans

• Paid time off

• Flexible work arrangements

• Professional development