QA Automation Tester

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee and sustain Databricks workspaces throughout development, testing, and production environments

• Set up and manage clusters, pools, jobs, and workflows for ETL, streaming, and machine learning operations

• Monitor the health, capacity, and performance of the platform and address issues proactively

• Administer Databricks runtimes, libraries, and integrations with cloud storage and data services

• Establish backup, recovery, and business continuity protocols

• Configure IAM roles, groups, and access permissions

• Implement security measures, encryption, network controls, and data access policies

• Collaborate with security and compliance teams to meet organizational and regulatory obligations

• Uphold data governance standards including Unity Catalog, catalogs, schemas, tables, and permissions

• Enhance cluster configurations, autoscaling, job scheduling, performance, and cost efficiency

• Analyze usage metrics and apply cost-control and chargeback/showback strategies

• Partner with data engineers, analysts, and data scientists to address workload requirements and Databricks best practices

• Provide Tier 2/3 support for Databricks incidents and service requests

• Create platform documentation, runbooks, and standard operating procedures

• Conduct user onboarding, training, and knowledge-sharing sessions

• Automate workspace and cluster provisioning, configuration, and policy enforcement utilizing Infrastructure as Code

• Integrate Databricks with CI/CD pipelines

• Support version control for notebooks, jobs, and configurations

• Acquire and maintain Public Trust clearance and complete onsite fingerprinting

⛳️ Requirements

• A minimum of 8 years of experience in managing cloud-based data platforms or big data environments

• Practical administration experience with Databricks

• Profound understanding of Databricks workspaces, clusters, jobs, pools, and notebooks

• Familiarity with cloud storage solutions such as Azure Data Lake Storage, S3, or GCS and networking concepts

• Experience with identity and access management in the applicable cloud provider

• Proficiency in Python, PowerShell, or Bash scripting and automation

• Knowledge of Spark concepts and performance considerations

• Experience in monitoring and resolving platform performance and reliability issues

• Strong problem-solving and analytical abilities

• Capacity to convey complex technical concepts to both technical and non-technical audiences

• Ability to work autonomously and collaboratively in a dynamic, agile environment

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related discipline

• If a relevant four-year degree is lacking, an additional four years of pertinent work experience is necessary

• Public Trust clearance required, including onsite fingerprinting at a designated facility

• U.S. residency requirement: must have resided in the U.S. or U.S. Territories for the specified recent periods

• Databricks certifications are preferred

• Experience with Unity Catalog, CI/CD tools, Infrastructure as Code, log aggregation and monitoring tools, Delta Lake, structured streaming, and data Lakehouse architectures is preferred

🏝️ Benefits

• Comprehensive options for medical, dental, and vision plans

• Health Savings Account options available

• 401(k) plan with both pre- and post-tax contributions and company matching

• Flexibility in work weeks where applicable

• Paid vacation, sick, personal, holiday, parental, military, bereavement, and jury duty leave

• Typically, 15 days of paid leave per calendar year

• 10 paid holidays annually

• Up to 160 hours of paid family leave within a rolling 12-month period for eligible employees

• Short- and long-term disability benefits

• Life insurance coverage

• Accidental death and dismemberment insurance

• Personal accident insurance

• Critical illness insurance

• Business travel and accident insurance

• AI-powered career tools and learning opportunities

• Support for internal mobility

• Wellness packages available

• Options for remote work

• Travel requirement is less than 10%