Senior Spark QA Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Conduct both manual and automated testing for Spark jobs, Spark SQL queries, and ETL pipelines.

• Implement functional, scalability, and performance testing for Spark workloads.

• Establish and oversee Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.

• Perform benchmarking, validation, and performance evaluation of Spark applications.

• Detect bottlenecks and resolve issues within distributed systems.

• Spearhead QA initiatives and provide mentorship to team members.

⛳️ Requirements

• A minimum of 5 years of QA/testing experience with substantial hands-on proficiency in Apache Spark.

• Background in functional and performance testing of distributed systems.

• In-depth knowledge of Spark architecture and optimization strategies.

• Familiarity with Kubernetes and cloud-based Spark environments.

• Proficient in Python or Java.

• Experience with automation frameworks and CI/CD pipelines.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and training.

• Flexible work hours and remote work options.

• A collaborative and inclusive work environment.