
Data Engineer
Posted 6 days ago

Posted 6 days ago
This is a fully remote position, open to applicants in Virginia.
• Design, develop, and manage scalable data ingestion and processing pipelines utilizing Azure Data Factory, Azure Databricks, Apache Kafka, and associated cloud technologies.
• Engineer and enhance data lakehouse solutions, which include Delta Lake architectures, semantic models, conformed dimensions, and partitioning strategies.
• Integrate data from enterprise systems, legacy platforms, and cloud-based sources through batch, streaming, and event-driven ingestion patterns.
• Implement automated data validation, reconciliation, and monitoring processes.
• Assist in migrating data pipelines and analytical workloads to Summit, ensuring parity with source systems and validating production readiness.
• Optimize data processing performance, resource usage, and cloud expenditure.
• Collaborate with data governance specialists to document datasets in the Enterprise Data Catalog, including metadata, definitions, and lineage.
• Enforce Government-approved data governance requirements, access controls, data quality standards, and dataset publication criteria.
• Develop and maintain comprehensive technical documentation.
• Monitor production pipelines, troubleshoot failures and performance issues, and apply corrective measures.
• Support the VA Summit ATLAS program and the modernization of VA's enterprise data environment.
• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related technical field, or a comparable mix of education and relevant professional experience.
• A minimum of seven years of professional experience in data engineering, data integration, or enterprise data platform development.
• Proven hands-on experience with Azure Data Factory, Azure Databricks, and Apache Kafka.
• Strong expertise in Structured Query Language (SQL) and Python for data processing, transformation, and pipeline development.
• Experience with designing and maintaining Extract, Transform, and Load (ETL) and Extract, Load, and Transform (ELT) pipelines.
• Proficient in cloud-based lakehouse architectures, distributed data processing, and data modeling.
• Experience with implementing automated data quality checks, pipeline monitoring, error handling, and performance enhancement.
• Skilled in integrating data governance requirements into engineering workflows, including metadata management, data lineage, and policy-based data access controls.
• Familiarity with source control, automated testing, and Continuous Integration and Continuous Delivery (CI/CD) practices.
• Ability to fulfill VA personnel vetting, background investigation, and system access requirements.
• Willingness to travel for 1-2 domestic business trips each year.
• Health, dental, and vision plans.
• Optional Flexible Spending Account (FSA).
• Paid parental leave.
• Safe Harbor 401(k) with employer contributions that are 100% vested from day one.
• Paid time off and 11 paid holidays.
• No-cost group term life/AD&D plan, with optional supplemental coverage available.
• Pet insurance.
• Monthly stipend for phone and internet expenses.
• Tuition and training reimbursement.
• Comprehensive medical/dental/vision insurance coverage.
• Supportive work/life balance.
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.