Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Assist in the IRS data platform initiatives within a Databricks and AWS environment.

• Oversee secure and scalable data architecture along with tenant onboarding processes.

• Direct workspace architecture, Unity Catalog governance, and the design of cluster policies for IRS tenant organizations.

• Conduct tenant discovery, gather requirements, perform source profiling, and classify security for new data intake requests.

• Create comprehensive technical designs for tenant onboarding, which includes Data Sharing Agreements and SLA documentation.

• Manage the Requirements Traceability Matrix (RTM) and prepare submissions for the Architecture Review Board (ARB).

• Contribute to the automation of tenant onboarding, integration of financial operations, and reporting of ROI.

• Produce tenant training materials and lead weekly office hours for IRS program teams.

• Provide support for P1/P2 incident responses within SLA for tenant-related platform issues.

• Supply NIST 800-53 control evidence within five business days of IRS audit requests.

• Collaborate with stakeholders to promote platform adoption and achieve operational excellence.

• Ensure adherence to federal security standards and IRS requirements.

⛳️ Requirements

• A valid Minimum Background Investigation (MBI) is required; candidates currently in process are not eligible.

• U.S. Citizenship is necessary.

• A minimum of 8 years of experience in cloud data engineering.

• 3 to 5 years of practical experience with Databricks, including Unity Catalog, Delta Live Tables, Auto Loader, and Genie.

• Must possess a current and valid Databricks Certified Data Engineer Professional certification or an advanced equivalent at the time of award.

• AWS Certified Solutions Architect or Data Engineer certification at either the Associate or Professional level is required.

• Extensive experience with medallion architecture (Bronze/Silver/Gold), Delta Lake, and CDC patterns.

• Proficient in Python, PySpark, and SQL for developing enterprise pipelines.

• Hands-on experience with AWS S3, IAM, VPC, CloudWatch, Glue, and Athena.

• Familiarity with RBAC/ABAC, row-level security, and PII column masking.

• Knowledge of IRS OneSDLC, FISMA, NIST 800-53, and FedRAMP High is required.

• A Bachelor's degree in Computer Science, Information Technology, or a related field is mandatory.

• Experience with tools and technologies such as Terraform, GitHub Actions or Azure DevOps, Kafka/MSK, dbt, Great Expectations, Splunk/Datadog, Collibra/Alation, AWS Lake Formation, and data mesh or modern data product architectures is preferred.

• Experience with machine learning pipelines, data governance frameworks, and real-time streaming systems is a plus.

• Additional cloud certifications like AWS Certified Data Analytics or Azure Data Engineer are advantageous.

• Candidates must meet eligibility requirements for access to classified information and be able to secure a government-granted security clearance.

• The position may require criminal history, employment and education verification, drug testing, and creditworthiness investigations.

• Employment is contingent upon contract award and funding; it is not guaranteed.

🏝️ Benefits

• Military Veterans are strongly encouraged to apply.

• Compensation decisions take into account location, skill sets, experience and training, security clearances, licensure and certifications, as well as other business and organizational needs.