QA Analyst, Databricks

This is a fully remote position, open to applicants in Chile.

📋 Description

• Design and establish a data quality framework across Bronze, Silver, and Gold layers, defining validation rules, threshold tolerances, and alerting standards.

• Develop and sustain automated data quality checks within Databricks pipelines, including row counts, null checks, referential integrity, schema validation, and business rule assertions.

• Manage reconciliation between source systems and Databricks layers, ensuring that source data is accurately ingested and that transformations yield expected outputs.

• Verify identity resolution outputs in the Silver layer by reviewing match rates, investigating false positives and negatives, and ensuring proper assignment of enterprise identifiers across source populations.

• Conduct end-to-end pipeline testing to confirm that data flows correctly from ingestion through the Gold layer and that downstream reporting outputs reflect accurate information.

• Collaborate with Data Engineers to define acceptance criteria for each sprint’s pipeline and data model deliverables prior to their promotion to production.

• Assist UAT with client business stakeholders, aiding them in validating that Gold layer outputs meet their reporting requirements.

• Document all QA processes, test results, and data quality findings in a format that can be transitioned to the client team at the conclusion of engagement.

• Monitor pipeline health after deployment, investigating and triaging data quality incidents and working alongside engineers to swiftly resolve root causes.

⛳️ Requirements

• Proven experience with Azure-based data platforms, including Databricks.

• Strong comprehension of data quality frameworks and testing methodologies for data pipelines.

• Experience in validating ETL/ELT processes and familiarity with layered architectures (Bronze, Silver, Gold).

• Proficient SQL skills and experience in analyzing large datasets.

• Background in implementing automated data validation and reconciliation processes.

• Understanding of data pipeline monitoring, alerting, and troubleshooting techniques.

• Ability to work collaboratively with Data Engineers and business stakeholders.

• Strong analytical capabilities and attention to detail.

• Experience in documenting QA processes and results in a systematic manner.

• Proficiency in English: Advanced (essential for effective communication with global teams).

🏝️ Benefits

• 📚Learning Opportunities: Certifications in AWS (we are AWS Partners), Databricks, and Snowflake.

• Access to AI learning paths to remain current with the latest technologies.

• Tailored study plans, courses, and additional certifications specific to your role.

• Access to Udemy Business, which offers thousands of courses to enhance your technical and soft skills.

• English lessons to facilitate your professional communication.

• 👨🏽‍💻Travel opportunities to attend industry conferences and connect with clients.

• 👩‍🏫 Mentoring and Development: Career development plans and mentorship programs designed to shape your career path.

• 🎁 Celebrations & Support: Special day rewards to commemorate birthdays, work anniversaries, and other personal milestones.

• Company-provided equipment.

• ⚖️ Flexible working arrangements to help you achieve a suitable work-life balance.

• Other benefits may vary based on your location in LATAM.