Data Engineer – Databricks

This is a fully remote position, open to applicants in California.

📋 Description

• Create and develop data pipelines for production using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming.

• Design and implement Lakehouse solutions utilizing Databricks.

• Construct and uphold transformation layers within Databricks.

• Develop and sustain the data and AI foundations that drive production machine learning.

• Work collaboratively with product and backend engineers to formulate data models.

• Engage with clients to comprehend their data-related challenges.

⛳️ Requirements

• 3-5 years of experience in data engineering, including a minimum of 2 years in production environments using Databricks.

• Strong understanding of AWS and Azure cloud services pertinent to Databricks implementations.

• Extensive, production-level expertise in Databricks: Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming.

• Demonstrated experience in designing Lakehouse architectures.

• Practical experience in data pipeline testing, observability, and CI/CD processes for data.

• High proficiency in SQL and Python programming languages.

• Familiarity with data modeling, schema design, and query optimization techniques.

• Exceptional communication skills, with the capability to articulate complex data concepts effectively.

🏝️ Benefits

• Health insurance

• Retirement plans

• Paid time off

• Flexible work arrangements

• Professional development