Lead Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee daily task prioritization and provide technical guidance to team members

• Conduct code reviews, uphold design best practices, and mentor engineers in modular Python development and SQL optimization

• Gather, refine, and convert intricate mission requirements into technical specifications, interface designs, and scalable data models

• Design, develop, and enhance high-throughput ETL/ELT workflows utilizing Databricks

• Sustain, refactor, and improve Python and SQL codebases for optimal performance, stability, and security

• Detect pipeline bottlenecks, troubleshoot complex data workflows, and implement automated solutions to enhance pipeline reliability and data integrity

• Collaborate with government product owners, data scientists, data analysts, and technical colleagues

• Create architecture documentation, coordinate technical sprints, and convey engineering trade-offs to both technical and non-technical audiences

• Establish and maintain documentation for pipeline architecture and configurations

⛳️ Requirements

• Must be a U.S. Citizen

• Must possess a current, active U.S. Government Secret clearance; applicants without an active Secret clearance will not be considered

• A minimum of 8+ years of professional experience in data engineering, constructing enterprise-scale data platforms and pipelines

• Proven experience in providing technical mentorship, conducting code reviews, and overseeing daily technical tasks for other engineers

• Demonstrated hands-on experience in designing and optimizing data transformation workflows using Databricks

• Advanced proficiency in modular, object-oriented Python/PySpark scripting

• Extensive experience in writing, tuning, and optimizing complex SQL queries

• Proven track record in creating functional, user-centric reports and dashboards with modern data visualization tools

🏝️ Benefits

• Comprehensive benefits programs

• Performance-based or alternative incentive compensation, where applicable

• Reasonable accommodations for qualified individuals with disabilities or disabled veterans