Senior Data Engineer

This is a fully remote position, open to applicants in District of Columbia, +1 more state.

📋 Description

• Design and develop reusable frameworks for data ingestion, transformation, orchestration, and validation that align with the Summit product roadmap and established enterprise architecture.

• Spearhead the engineering of shared pipeline templates, data-quality gates, lineage instrumentation, and automated deployment or recovery strategies.

• Enhance cloud data processing efficiency, partitioning, performance, reliability, and cost; set technical metrics and tuning recommendations.

• Offer technical expertise on intricate data integration, source parity, semantic modeling, and federated query functionalities.

• Work in collaboration with architecture and governance leaders.

• Incorporate automated testing, observability, CI/CD, infrastructure-as-code, and secure access controls into data engineering workflows for production.

• Diagnose complex cross-platform failures and data quality challenges.

• Oversee root-cause analysis, remediation, rollback, and operational readiness procedures.

• Maintain comprehensive design documentation, standards implementation guidance, runbooks, and technical decision logs.

• Mentor engineers and work with customer-requested engineering teams to encourage the adoption and reuse of common capabilities.

• Facilitate collaboration during Eastern Time hours and provide applicable production coverage.

• Offer occasional after-hours or weekend support for critical outages or scheduled maintenance.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related technical field, along with 10+ years of relevant experience.

• Proven expertise in designing and implementing complex data engineering solutions.

• Practical experience in developing and troubleshooting production data ingestion, ETL/ELT, transformation, and validation pipelines using SQL and Python.

• Familiarity with cloud-based data platforms and services such as Microsoft Azure, AWS, or Google Cloud.

• Understanding of storage, orchestration, and secure data access methodologies.

• Experience in data modeling, source-to-target reconciliation, data quality testing, and pipeline monitoring or incident troubleshooting.

• Capability to document technical designs, data flows, dependencies, and operational procedures.

• Ability to collaborate effectively with engineers, analysts, and data owners.

• Demonstrated experience in creating reusable production data engineering frameworks or shared services.

• Proficiency in independently resolving complex integration and performance issues.

• Experience enhancing data engineering delivery through automation, testing, CI/CD, or standardized deployment practices.

• Ability to obtain and maintain the necessary VA Public Trust suitability determination (Tier 2/Moderate Risk).

• Preferred: Direct, hands-on experience supporting the VA Summit Data Platform.

• Experience with large-scale Spark/Databricks or Delta Lake processing and optimizing performance or cost across platforms.

• Experience mentoring engineers or coordinating shared engineering capabilities across various delivery teams.

• Practical knowledge of Microsoft Azure data engineering, including Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Spark, or similar services.

• Familiarity with Delta Lake/lakehouse architectures, data catalogs and lineage, and governance tools such as Collibra, Immuta, or Starburst.

• Experience in supporting federal IT systems while adhering to security, privacy, and data access standards.

• Experience with Agile delivery methodologies and relevant cloud/data engineering certifications.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health, dental, and vision insurance packages.

• Retirement savings plan with employer matching contributions.

• Opportunities for professional development and continuous learning.

• Flexible work schedule and remote work options.

• Generous vacation and leave policies.