Senior Databricks Engineer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Design and develop scalable data solutions utilizing the Databricks Lakehouse architecture.

• Create and enhance batch and real-time data pipelines for large-scale data processing.

• Lead the technical design of solutions for data integration, transformation, and analytics tasks.

• Collaborate with business stakeholders, data engineers, analysts, and product teams to convert requirements into technical solutions.

• Enforce data governance, security, and quality best practices through Unity Catalog and related technologies.

• Improve performance, reliability, and cost-effectiveness of Databricks workloads.

• Assist in modernizing data warehouses and initiatives for cloud migration.

• Produce architecture documentation, technical standards, and design guidelines.

• Diagnose complex issues within the data platform and offer technical recommendations.

• Stay updated on Databricks and cloud platform capabilities while promoting the adoption of best practices.

• Provide high-quality data platforms that support analytics, reporting, and operational decision-making.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related discipline.

• At least 5 years of experience in data engineering, data architecture, or similar technical roles.

• Practical experience with Databricks in corporate environments.

• Strong proficiency in PySpark, Spark SQL, Python, and SQL.

• Experience in designing and implementing Lakehouse architectures.

• Proven ability to build scalable ETL/ELT data pipelines.

• Deep understanding of data modeling, data warehousing, and analytics platforms.

• Experience handling both structured and unstructured data.

• Excellent problem-solving and communication abilities.

• Capability to work independently in a remote setting.

• Experience in implementing data governance and security frameworks.

• Databricks Certified Data Engineer Associate certification is required.

• Databricks Certified Data Engineer Professional certification is required.

• Preferred experience in the healthcare industry, including claims, clinical, provider, member, or revenue cycle data.

• Preferred experience with Azure, AWS, or GCP cloud platforms.

• Preferred exposure to machine learning and AI/ML workloads on Databricks.

• Preferred cloud platform certifications.

🏝️ Benefits

• Full-time remote work opportunity.

• Individual contributor role.

• Chance to engage in healthcare analytics and modern data architecture projects.