Databricks Platform Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Construct and manage Databricks environments across development, testing, and production stages.

• Configure and enhance Databricks workspaces, clusters, and job orchestration processes.

• Execute Infrastructure-as-Code using Terraform, CloudFormation, or similar tools.

• Create automation for platform provisioning, monitoring, scaling, and maintenance activities.

• Facilitate CI/CD pipelines for Databricks notebooks, workflows, and deployments.

• Oversee platform health, performance, security, and cost optimization efforts.

• Apply RBAC, encryption, audit logging, and governance controls.

• Diagnose platform issues, cluster performance challenges, and workload failures.

• Assist in integration with cloud services, data warehouses, and streaming platforms.

• Collaborate with data engineers, data scientists, and cloud infrastructure teams.

⛳️ Requirements

• Minimum of 5 years of experience in platform engineering, cloud infrastructure, or IT operations.

• At least 2 years of practical Databricks administration experience.

• Proficient in Databricks Workspace Administration.

• Strong knowledge of Apache Spark.

• Expertise in programming with Python, Bash, or PowerShell.

• Extensive experience with Infrastructure-as-Code (Terraform, CloudFormation, ARM).

• Experience in supporting Azure Government, AWS GovCloud, or other secure cloud environments.

• Understanding of cloud networking, security, and platform monitoring.

• Experience in implementing RBAC, encryption, audit logging, and security measures.

• Familiarity with federal compliance frameworks such as FedRAMP, FISMA, and NIST 800-53.

• Strong troubleshooting skills and platform support experience.

• Must be eligible to obtain and maintain a Public Trust clearance or higher.

• Databricks Certified Associate or Platform Administrator certification is preferred.

• Experience in Unity Catalog administration and data governance is preferred.

• Knowledge of CI/CD automation and DevOps best practices is preferred.

• Experience with Kafka, AWS Kinesis, Azure Event Hubs, or other streaming technologies is preferred.

• Familiarity with Snowflake, Redshift, BigQuery, or enterprise data warehouse platforms is preferred.

• Experience in optimizing Databricks clusters for performance and cost efficiency is preferred.

• Federal cloud certifications such as Azure Administrator Associate or AWS Solutions Architect are preferred.

• Prior experience supporting federal agencies and working in Public Trust environments is preferred.

🏝️ Benefits

• Equal Employment Opportunity / Affirmative Action employer.

• Opportunity to contribute to a 5-year performance program.

• Remote work arrangement available.