Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, construct, deploy, and maintain scalable ETL/ELT pipelines utilizing SQL, Spark, Databricks, and Azure-native tools.

• Implement workflow orchestration, scheduling, and monitoring to ensure reliable automated data delivery.

• Develop tools for data quality monitoring and observability to track lineage and maintain data integrity.

• Optimize data ingestion, transformation, and storage for enhanced performance, cost-effectiveness, and maintainability.

• Administer and enhance relational databases including SQL Server, Azure SQL, and PostgreSQL.

• Conduct database performance tuning, indexing, query optimization, and capacity planning.

• Implement and uphold database security, access controls, and auditing in compliance with VA requirements.

• Support strategies for backup/restore, disaster recovery planning, and high-availability configurations.

• Monitor database health, troubleshoot problems, and ensure data integrity across different environments.

• Perform root-cause analysis and continuously enhance data engineering and DBA processes.

• Implement automated data quality checks, validation rules, and anomaly detection routines.

• Develop monitoring dashboards and alerting systems to proactively identify data issues.

• Assist in data profiling, lineage tracking, and metadata management.

• Collaborate with stakeholders to establish quality thresholds and acceptance criteria.

• Translate VA stakeholder requirements into scalable, repeatable data engineering and database solutions.

• Work alongside developers, data scientists, and program stakeholders to deliver dependable, high-performance data and analytics products.

⛳️ Requirements

• Bachelor’s degree.

• Over 5 years of experience in data engineering, ETL/ELT development, or database administration.

• Strong proficiency in SQL, Python, PySpark, Spark, and Databricks.

• Experience in performance tuning, indexing, and query optimization.

• Experience in administering and optimizing relational databases in production settings.

• Familiarity with working across VA platforms and enterprise data assets.

• Experience in implementing automated data pipelines and data quality monitoring frameworks.

• Knowledge of Azure Data Factory, Azure Synapse, Azure Data Lake Storage, and Azure SQL.

• Experience with GitHub, GitLab, and GitHub Actions.

• Must be authorized to work in the U.S. indefinitely without sponsorship.

• Ability to obtain a public trust.

• Preferred: experience with large-scale unstructured datasets, data platforms and processing tools, legacy/on-premises-to-cloud migrations, medallion architecture, data governance and quality management, and certifications in Amazon or Azure cloud.

🏝️ Benefits

• Health, dental, and vision coverage.

• Flexible spending accounts.

• Disability and life insurance.

• Retirement plan.

• Paid time off.

• Additional programs to support employees and their families.

• Reasonable accommodations available during the application or interview process.

• Potential client-specific or government-mandated workplace health and safety provisions.