Lead Data – ETL Engineer

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Oversee the design, execution, maintenance, and enhancement of enterprise data integration and ETL pipelines.

• Maintain and enhance SSIS packages, including their configurations, schedules, dependencies, and execution processes.

• Create SQL queries, stored procedures, transformations, and reconciliation logic.

• Utilize Python or another authorized automation language for ingestion, orchestration, validation, monitoring, and recovery tasks.

• Execute API integrations, covering aspects such as authentication, pagination, rate-limit management, retries, and error processing.

• Develop incremental loading strategies and implement change-data capture where applicable.

• Construct and sustain raw, standardized, and curated data layers with clearly documented transformation rules.

• Define source-to-target mappings, common identifier mappings, and reference data management.

• Establish reconciliation checks for record counts, totals, completeness, and transformation accuracy.

• Create restartable processing, controlled retries, checkpoints, and recovery protocols.

• Manage failed-record handling, quarantine procedures, error logging, and controlled reprocessing.

• Evaluate and address schema alterations and their effects on pipelines, datasets, analytical models, and reports.

• Maintain data lineage, transformation documentation, interface specifications, and pipeline dependencies.

• Monitor pipeline execution, throughput, freshness, resource usage, and operational failures.

• Diagnose complex issues involving source systems, APIs, SSIS packages, SQL logic, data quality, and downstream integrations.

• Enhance pipeline performance while ensuring data integrity, traceability, and compliance with approved security measures.

• Collaborate with governance personnel on definitions, ownership, authoritative sources, metadata, and sanctioned data usage.

• Assist with reconciliation, regression testing, defect resolution, and release validation alongside independent testing teams.

• Coordinate pipeline modifications with database, application, AI/ML, warehouse, and BI teams.

• Maintain version-controlled code, deployment instructions, rollback strategies, and operational runbooks.

• Evaluate technical designs and code, mentor engineering practices, and facilitate knowledge transfer to other engineers.

• Communicate pipeline status, risks, dependencies, corrective measures, and enhancement suggestions to technical leads and Government stakeholders.

⛳️ Requirements

• Active Secret clearance or higher.

• At least 12 years of relevant professional experience.

• Bachelor’s degree in computer science, information systems, or a related field, or a recognized equivalent.

• Proficient in maintaining and optimizing SQL Server Integration Services (SSIS).

• Strong SQL proficiency.

• Experience with Python and data engineering automation.

• Background in implementing and supporting API integrations.

• Experience in developing incremental data loads.

• Familiarity with documenting and maintaining data lineage.

• Skilled in troubleshooting complex data pipelines.

• Capability to build raw, standardized, and curated data layers.

• Proficient in managing reconciliation, retries, schema changes, identifier mappings, and failed-record handling.

• Ability to implement change-data capture where suitable.

• CompTIA Security+ CE or another approved IAT Level II certification, as applicable to assigned duties.

🏝️ Benefits

• Regular full-time employment.

• Opportunity to provide technical leadership and hands-on engineering on a USMEPCOM ENLIST government program.

• Knowledge transfer and technical mentoring opportunities.

• Equal employment opportunity and affirmative action protections.