Senior Data Engineer – Databricks, Security, Governance

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the design and execution of a unified data schema for federal species information within a cloud data lakehouse.

• Address structural and semantic inconsistencies across various source systems.

• Create documentation for field-level source-to-target mapping, detailing transformation logic, data type conversions, null management, and data lineage.

• Establish and uphold metadata standards for all ingested datasets.

• Develop and implement automated frameworks for data quality validation within ETL pipelines.

• Construct the Metadata and Authoritative Source Registry.

• Apply FIPS 140-compliant encryption measures and enforce NIST SP 800-171/800-172 database security controls along with role-based access management.

• Incorporate Federal CUI Registry and data governance standards into the platform's data.

• Direct the monitoring and optimization of database performance within the cloud data platform.

• Contribute to the Data Model Harmonization Report with database-layer schema decisions, field-level mapping documentation, and metadata definitions.

• Manage database-related components of the production migration runbook and carry out database migration tasks during the production cutover.

• Record migration results, post-deployment data validation findings, and production monitoring insights in the Production Deployment Report.

• Oversee the documentation of the database for Final Operational Playbooks.

⛳️ Requirements

• Bachelor's degree with 12 years of professional experience, a Master's degree with 10 years of professional experience, an Associate's degree with 14 years of professional experience, or a high school diploma/equivalent with 16 years of professional experience.

• U.S. Citizenship is required.

• Must possess the ability to obtain and maintain a Public Trust clearance.

• Extensive experience in database engineering, particularly with cloud data platforms.

• Hands-on experience with Databricks, Delta Lake, or similar cloud data lakehouse platforms.

• Proficient in Python or Scala for implementing ETL pipelines, developing data quality frameworks, and creating database automation scripts.

• Familiarity with FIPS 140, NIST SP 800-171/800-172, and federal CUI data governance standards.

• Experience in designing and implementing role-based access control (RBAC) and data security measures in federal cloud environments.

• Proven experience in leading database migrations and overseeing production cutover activities for federal programs.

• Experience with version-controlled code repositories and enforcing peer review processes.

• Skilled in documenting post-deployment monitoring results and production validation outcomes.

• Strong abilities in data modeling, schema design, and metadata management.

🏝️ Benefits

• Employees may qualify for overtime compensation.

• Employees may be eligible for shift differentials.

• Employees may receive a discretionary bonus.

• Equal opportunity employer, including individuals with disabilities and protected veterans, or other characteristics safeguarded by law.